An Energy-Efficient Unstructured Sparsity-Aware Deep SNN Accelerator With 3-D Computation Array

Deep spiking neural networks (DSNNs), such as spiking transformers, have demonstrated comparable performance to artificial neural networks (ANNs). With higher spike input sparsity and the utilization of accumulation (AC)-only operations, DSNNs have great potential for achieving high energy efficiency. Many researchers have proposed neuromorphic processors to accelerate spiking neural networks (SNNs) with dedicated architectures. However, three problems still exist when processing DSNNs, including redundant memory access among timesteps, inefficiency in exploiting unstructured sparsity in spikes, and the lack of optimizations for new operators involved in DSNNs. In this work, an accelerator for deep and sparse SNNs is proposed with three design features: a 3-D computation array that allows parallel computation of multiple timesteps to maximize weight data reuse and reduce external memory access; a parallel non-zero data fetcher that efficiently searches non-zero spike positions and fetches corresponding weights to reduce computation latency; and a multimode unified computation scheduler that can be configured to maximize energy efficiency for spiking convolution (SCONV), spiking <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q, K,~\text {and}~V$ </tex-math></inline-formula> matrix generation, and spiking self-attention (SSA). The accelerator is implemented and fabricated using 40-nm CMOS technology. When compared with state-of-the-art sparse processors, it achieves the best energy efficiency of 0.078 pJ/SOP and the highest recognition accuracy of 77.6% on ImageNet using the spiking transformer algorithm.