SVRoM: A 9.52-mW Video Understanding Smart Vision SoC With On-Chip Sensing and Similarity-Aware SRAM/ROM CIM Macro

The rapid growth of virtual reality (VR), augmented reality (AR), extended reality (XR), and intelligent surveillance systems has driven increasing demand for video understanding tasks on edge devices. However, supporting such tasks on edge devices remains challenging due to the limited computational resources, memory constraints, power budgets, and high data transition latency. To address these challenges, this work proposes an ultralow-power smart vision System-on-Chip (SoC) with the following features: 1) a bitline segmented, parallel-charging read-only memory (ROM) compute-in-memory (CIM) macro with power gating and coupled coding; 2) heterogeneous similarity-aware hybrid SRAM/ROM cores to exploit temporal similarity with high core utilization; 3) an intracore and intercore pipeline with hierarchical dataflow for low-power data transmission; 4) a tilewise mixed-precision weight quantization and mapping scheme for weight data compression; and 5) a hierarchical multimodal system trigger mechanism utilizing an on-chip CMOS imager with near sensor caching to skip unnecessary inferences. The proposed ROM CIM macro achieves an area efficiency of 0.753–1.673 TOPS/mm<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> and a storage density of 4706 Kb/mm<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>, achieving a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3.2\!\!-\!\!14.83\times $</tex-math> </inline-formula> improvement in density Figure of Merit (FoM) over the state-of-the-art (SoTA) ROM CIM designs. The proposed SoC also shows an ultralow always-on power of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0.13~\mu $</tex-math> </inline-formula>W and an average active power of 9.52 mW, which demonstrates a high energy efficiency of 13.6–18.3 TOPS/W (ResNet-20 with fixed-point 8-bit activation and 10-bit weight) on video understanding tasks, showing a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.8-3.0\times $</tex-math> </inline-formula> improvement over the existing smart vision SoCs.