A Brain-Inspired Intelligent Vision Sensor With Spiking Processing-in-Sensor Architecture for Spike Feature Encoding

The relentless growth of artificial intelligence of things (AIoT) applications has tightened sub-milliwatt power-budget constraints on embedded vision systems. To address this challenge, this brief presents the first brain-inspired intelligent vision sensor (BIVS) based on a novel spiking processing-in-sensor (PIS) architecture that efficiently converts captured pulsewidth-modulation (PWM) image data into real-time sparse spike features for subsequent intelligent processing, offering an attractive balance between computational accuracy and energy efficiency. A <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$126\times 126$ </tex-math></inline-formula> BIVS is designed in 180nm CMOS technology and consumes <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$66.9\mu $ </tex-math></inline-formula>W at 661fps with a resulting efficiency of 5.02 TOPS/W and FoM of 6.37 pJ/pixel/frame. It also achieves a spike feature map accuracy of 99.73% on the MNIST dataset and a classification accuracy of 99.05% when coupled with a simplified spiking convolutional neural network (SCNN).