A 28nm 166.9 TOPS/W x Mb/mm² DRAM-Free QLC Compute-in-ROM Macro Supporting High Task-Level Inference Energy Efficiency for Tiny AI Edge Devices

Compute-in-Memory (CiM) is recognized as a promising approach to mitigating the high communication costs due to the “memory wall” bottleneck in the traditional von Neumann architecture. Given the advancements in the Internet-of-Things (IOT) and artificial intelligence (AI), where neural network (NN) models are continually increasing in size and complexity, large storage CiM holds promise for accelerating deep NN (DNNs) on resource-constrained edge devices, enabling efficient processing with minimal or no weights loading from off-chip memory. However, as illustrated in Fig. 1, there are two primary challenges that hinder existing CiM solutions [2–6] for edge Al applications: (1) the high cost of off-chip data movement due to low on-chip CiM macro density, and (2) the trade-off between CiM output A/D conversion (ADC) precision, energy, and throughput in multi-bit multiply-and-accumulate (MAC) operations.