A 5.9 μJ/Inference Reconfigurable Accelerator for Hybrid SNN-ANN Architectures in Event-based Image Recognition

Event-based images generated by event-based cameras record a stream of events with spatiotemporal information, which is different from static images. Although spiking neural networks (SNNs), which are event-based models, can be used to process event-based images naturally, they suffer from low efficiency in the training, compared to artificial neural networks (ANNs). Hybrid SNN-ANN architectures are a promising alternative to achieve both high performance and low latency. In this paper, we propose an efficient edge accelerator for hybrid SNNANN architectures. A fused reconfigurable PE array is proposed to support both SNN layers with time-parallel dataflow and ANN layers. An offline output rearrangement scheme is further proposed to improve the sparsity utilization of the ANN layers with low performance degradation and extra cost. The proposed edge accelerator is implemented with 28 nm CMOS technology, running at 400 MHz. The accelerator achieves the accuracy of 94.44% on DVS gesture dataset, and a throughput of $6,711 \mathrm{inf} / \mathrm{s}$ with $5.9 \mu \mathrm{~J}$/inf energy consuming.