A 1.1 μJ/Inference Binary Spiking Neural Network Accelerator for DVS Gesture Recognition

Dynamic vision sensors (DVS) are bioinspired sensors that can generate sparse data streams with low latency, low power consumption and high dynamic range. Spiking neural networks (SNNs), which are inspired by biological brains, are event-based models, and therefore they can be used to process the binary data streams produced by such sensors naturally. However, SNN accelerators usually require more memory and longer time for inference, due to the extra time dimension in SNNs. In this paper, an energy efficient binary spiking neural network (BSNN) accelerator for DVS gesture recognition is proposed with algorithm and hardware codesign. We integrate the binary neural network (BNN) training method into SNN to directly train a BSNN model, which significantly reduces memory consumption. A temporal pooling (TP) layer is further proposed to reduce the time steps in SNNs while maintaining competitive accuracy. The proposed BSNN accelerator can achieve high parallelism with high resource utilization, and the sparsity of input spikes is utilized to further reduce power consumption. The proposed BSNN model achieves an accuracy of 95.49% on IBM DVS Gesture dataset. The implementation results show that the BSNN accelerator can achieve 38.2k inference per second with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.1~\mu $</tex-math> </inline-formula>J/inference energy consumption and 216.9 TOPS/W energy efficiency.