Energy-Efficient Inference With Software–Hardware Co-Design for Sustainable Artificial Intelligence of Things

The emerging field of Artificial Intelligence of Things (AIoT) is propelled by the remarkable success of deep learning and hardware evolution, which has a significant impact on our daily lives. However, because of their notorious computing resource intensity, the widespread deployment of AIoT devices requires substantial electricity consumption as support, inevitably escalating energy consumption, and ultimately leads to a significant carbon emissions. Existing research on neural network compression and acceleration struggles to achieve energy-efficient inference on resource-constrained AIoT devices. To address this issue, we propose a software-hardware co-design approach that integrates advanced neural network optimization techniques with hardware power management capabilities to enable energy-efficient inference and ultimately achieve sustainable AIoT. We introduce a lightweight split and refinement block that adaptively reduces redundant computation in both channel and spatial dimensions. Several early exit (EE) branches are added to the backbone, which are controlled by a policy-based EE predictor. With the predicted EE index, a curve-fitting-based frequency scaling algorithm is presented to calculate the optimal frequency that minimizes energy overhead while maintaining latency constraints. Extensive experiments on CIFAR and CINIC classification tasks validate that our proposed method consistently reduces energy consumption for neural network inference while outperforming other competitive methods.