Improving Accuracy and Efficiency in DNNs with Approximate Multipliers: Insights from Information Bottleneck Theory

Approximate multipliers have potential to improve energy efficiency in Deep Neural Networks but introduce computational errors that degrade accuracy. This paper introduces a novel method, leveraging approximate multipliers to enhance accuracy, while improving computational and energy efficiency. We propose a layer-wise heterogeneous approach using quantized approximate multipliers (INT8) in DNNs, applying varying levels of approximation across layers. This approach achieves estimated energy savings of up to 44.75% across all evaluated configurations, including up to 43.72% for the VGG models, while improving Top-1 accuracy by up to 2.07 percentage points relative to the corresponding exact INT8 baseline. Using Information Bottleneck (IB) theory, we analyze the enhanced information flow and feature extraction capabilities enabled by approximate multipliers. Through Information Plane (IP) analysis, we gain insights into DNN behavior and demonstrate how this method can overcome accuracy limitations. An analytical MAC-count comparison indicates that, under the experimental settings considered, the forward-pass-only GA search requires 146× to 22,500× fewer MAC operations than the corresponding gradient-based training schedules; this comparison reflects computational workload rather than an equivalent-objective algorithmic speedup.