A Reconfigurable Digital Compute-In-Memory Heterogeneous Macro for Differential Frame Convolution and Spiking Neural Network

The application of artificial neural network (ANN) in video processing encounters significant challenges, including large data volumes, numerous linear operations, and high power consumption. The fusion of convolutional neural network (CNN) and spiking neural network (SNN) provides a dual benefit of achieving high accuracy while maintaining low power consumption. However, ongoing challenges remain in minimizing multiply-accumulate (MAC) operations and optimizing data movement. In this work, we propose a reconfigurable digital compute-in-memory (RDCIM) heterogeneous macro without the sense amplifier, tailored for the diverse computational demands of CNN and SNN. To improve energy efficiency, differential frame convolution (DFC) is adopted to mitigate the computational overhead. In addition, computational resources are functionally reused to accommodate four data flow types, supporting both DFC and SNN operations. Implemented by TSMC 28nm technology, the proposed RDCIM heterogeneous macro achieves the peak energy efficiency of 29.13 TOPS/W for DFC and 0.56 pJ/SOP for SNN, operating at a frequency of 284 MHz.