Low Power AI Inference Through Filter Sorting for Reducing Switching Activity of DNN Model

Achieving energy efficiency is of paramount importance in facilitating the deployment of artificial intelligence (AI) applications, particularly for edge devices. The computational complexity and memory requirement of AI workloads have made energy efficiency in circuits the primary design constraint, prompting a need for architectural reconsideration. However, operations in Convolutional Neural Networks (CNN) relying on matrix multiplication offer opportunities for algorithm-level optimization. The dynamic power consumption in CMOS technology exhibits a direct correlation with the switching activities (SA) associated with transitions of data bits from one cycle to the next. Minimizing SA constitutes an effective approach to curbing power consumption, resulting in prolonged battery life. This work introduces a cross-layer end-to-end optimization technique, which involves a one-time sorting of pre-trained model parameters to reduce switching activity during matrix multiplication or convolution operations while eliminating the indexing overhead during inference. While upto 40% reduction in switching activities through sorting has been achieved, the actual saving in dynamic power achievable for the input stationary systolic array is approximately 10% in total power consumption and around 15% in switching power. The proposed method has been validated using various pre-trained networks, including GoogeLeNet, MobileNet, AlexNet, and SqueezeNet, among others.