ScanNow: A Scan Window-Based Sparse Matrix Multiplication Accelerator Design

Sparse matrix-matrix multiplication (SpMM) is a prevailing kernel in scientific and artificial intelligence applications. However, the irregular memory access behaviors caused by diverse sparse patterns in SpMM lead to a significant performance bottleneck for traditional computing platforms, driving the demand for dedicated hardware accelerators. Unfortunately, the existing hardware accelerators are not flexible enough to handle diverse SpMM workloads due to their rigid task dispatch schemes, limiting the overall performance.To address the challenge, this paper introduces ScanNow, a scan window-based SpMM accelerator design. ScanNow first presents a scan window-based task dispatch scheme designed to minimize the time-consuming irregular memory access behaviors by exploiting potential data reuse opportunities. The proposed task dispatch scheme effectively and dynamically balances input and output data reuse for row-wise dataflow according to real-time sparse patterns. Moreover, a tailored hardware architecture, featuring multiple memory components and customized schedulers, is proposed to accommodate the task dispatch scheme and improve computation parallelism. Experiments demonstrate that ScanNow achieves an average speedup of 1.67x across a wide range of SpMM workloads compared to the state-of-the-art hardware accelerator.