Kung-Fu: An Energy-Efficient Compute-In-Memory Approach for Neural Network Inference Using Multi-Level Binary Computing Fusion

Compute-In-Memory (CiM) is an emerging architecture designed to address the memory wall issue in deep neural network (DNN) inference. However, both the ADC in analog CiM (ACiM) and the adder trees in digital CiM (DCiM) contribute to significant energy and area overhead. In response to these challenges, binary neural networks (BNNs) have been proposed recently. Nevertheless, accuracy degradation poses a serious challenge to the application of BNNs in CiM due to errors in partial-sum accumulations. Furthermore, post-processing steps involving binary activation, such as ReLU, scaling, and bias addition, introduce redundant computing that cannot be effectively optimized by BNN-CiM.This work proposes a novel software-hardware co-optimization approach aimed at enabling an ADC-free analog CiM design while maintaining accuracy. Multi-Level binary computing fusion techniques comprising redundant load isolation based row fusion, in-array parallelism adaption based block fusion, and high-precision post-process elimination based layer fusion address the serious accuracy issues associated with conventional BNN algorithms. In contrast with past over 10% accuracy lost BNN-CiM on practical dataset CIFAR-10 and ImageNet, this work achieves more than 2.2x energy efficiency and 7.4x memory density than state-of-the-art with only 2% accuracy loss.