Dyn-Bitpool: A 28 nm 27 TOPS/W Two-Sided Sparse CIM Accelerator Featuring a Balanced Workload Scheme and High CIM Macro Utilization

Deep neural networks (DNNs) have brought about a transformative impact across various sectors. However, the proliferation of DNNs has led to a surge in computational intensity and data traffic, thereby imposing substantial demands on the power capacity and battery life of computing systems. Computing-in-memory (CIM) is considered a promising architecture to resolve or mitigate the memory wall challenge by integrating computational elements within memory arrays. Yet prior studies on CIM have seldom capitalized on sparsity in both activations and weights simultaneously. Furthermore, the exploitation of two-sided sparsity—sparsity in both activations and weights—presents new challenges, such as imbalanced workload and low hardware substrate utilization. To harness the full potential of two-sided sparsity for acceleration, we present Dyn-Bitpool, an accelerator that introduces innovations on two fronts: 1) a balanced workload scheme, “pool first and cross lane sharing”, which maximizes performance gains enabled by the bit-level sparsity in activations; and 2) a dynamic topology for CIM arrays to effectively address the low CIM macro utilization issue caused by the value-level sparsity in weights. These collective advancements yield an average speedup of 1.91x and 2.67x for Dyn-Bitpool on eight prevalent neural networks, outperforming two cutting-edge CIM-based accelerators.