Sorted Weight Sectioning for Energy-Efficient Unstructured Sparse DNNs on Compute-in-Memory Crossbars

We introduce sorted weight sectioning (SWS): a weight allocation algorithm that places sorted deep neural network (DNN) weight sections on bit-sliced compute-in-memory (CIM) crossbars to reduce analog-to-digital converter (ADC) energy consumption. Data conversions are the most energy-intensive process in crossbar operation. SWS effectively reduces the ADC cost by leveraging (1) small weights and (2) zero weights (weight sparsity) present in DNNs.DNN weights follow bell-shaped distributions, with most weights near zero. Under SWS, we only need low-order crossbar columns for sections with low-magnitude weights. This reduces the quantity and resolution of ADCs required without significantly degrading DNN accuracy.Unstructured sparsification further sharpens the weight distribution with small accuracy loss. However, it presents challenges in hardware tracking of zeros: we cannot switch zero rows to other layer weights in unsorted crossbars without index matching. SWS uses offline remapping of zeros into earlier sections to exploit full sparsity potential and maximize energy efficiency.SWS reduces ADC energy use by 89.5% on unstructured sparse BERT models. Overall, this paper introduces a novel algorithm to allow energy-efficient CIM crossbars for unstructured sparse DNN workloads.