Exploring Sparse Inference with SuiteSparse:GraphBLAS

As AI models grow in size and complexity the re-source demands are also growing. Sparse inference is an approach to take advantage of sparsity in large AI models to make them run faster thereby lowering the resource requirements. SuiteS-parse: GraphBLAS, an open-source implementation of Graph-BLAS, offers a robust framework tailored for leveraging sparsity in matrix computations. This work explores the application of SuiteSparse:GraphBLAS in performing inference tasks, particularly focusing on the language model BERT. By showcasing the capabilities of SuiteSparse:GraphBLAS in handling sparse computations and explaining its underlying formulation using semirings, this explores its application in improving inference efficiency. This work implements a complete inference pipeline using SuiteSparse:GraphBLAS, compares its performance with traditional frameworks like PyTorch, and identifies areas for improvement. Through this investigation, the study aims to highlight the strengths and limitations of SuiteSparse:GraphBLAS in AI computations.