Voltage controlled magnetic tunnel junction based ADC less processing-in-pixel for extreme edge intelligence

The growing data generated by camera sensors has driven the need for energy-efficient solutions to deploy computer vision tasks on edge devices. Processing-in-pixel, which integrates parallel computation within the pixel array, improves energy and bandwidth efficiency by generating output activations of the first neural network layer rather than raw sensor data. This paper proposes an ADC-less processing-in-pixel architecture that implements a binary activation neural network optimized with a Hoyer regularizer for high accuracy in complex vision tasks. Additionally, a global shutter burst memory read scheme is introduced, enabling fast, disturbance-free read operations with nanoscale voltage-controlled magnetic tunnel junctions (VC-MTJs). The proposed algorithmic framework incorporates device and circuit constraints based on fabricated VC-MTJ characteristics and simulations using GlobalFoundries’ 22nm FDX technology. Evaluations on CIFAR10 and ImageNet datasets show 5.04 TOPS/W energy efficiency and a 6× reduction in bandwidth compared to traditional CMOS sensors, without any significant drop in test accuracy.