A 1.12nJ/Pixel High-Accuracy and Memory-Efficient Real-Time Object Detection Processor for Neuromorphic Vision Sensors

Neuromorphic vision sensors (NVSs) [1], which generate an event only when a pixel’s log intensity changes, feature low data redundancy and transmission latency, making them increasingly popular in various edge applications, such as traffic monitoring and object detection. Compared to event-by-event processing paradigms that require the processor to be always-on [2], frame-based counterparts are more power-efficient, as they keep the processor in an idle state during frame construction, resulting in a reduced signal toggling rate. However, as NVSs continue to evolve, the increasing resolution necessitates a substantial amount of on-chip memory to hold the reconstructed event-based binary image (EBBI), as depicted in Fig. 1 (top). Furthermore, most existing systems are suited only for stationary scenes, as moving NVSs introduce background noise [3]. In addition, prior object detection systems based on deep neural networks (DNN) or connected component labeling (CCL) suffer from significant memory overhead and long processing latency due to large model sizes and high computational intensity.