A Multi-Level Lightweight Framework for Deep Neural Networks on Resource-Constrained Edge Devices

Deep neural networks (DNNs) are widely used in artificial intelligence (AI) applications due to their powerful representational capacity. However, deploying DNNs on edge devices faces severe challenges, including limited computational resources, restricted memory capacity and heterogeneous processor architectures, which often make conventional DNN models infeasible. In this paper, we propose a novel multi-level model lightweighting framework designed to address these challenges. The framework integrates three key techniques: Neural architecture search (NAS), knowledge distillation (KD) and layer-adaptive quantization. NAS is employed to automatically discover an efficient network architecture under hardware constraints, enabling structural compression. Subsequently, KD transfers knowledge from the original model to the compressed model, mitigating the accuracy loss resulting from compression. Finally, a layer-adaptive quantization strategy is implemented, assigning distinct bit-widths to different layers based on their data distribution characteristics, thereby enhancing storage and computational efficiency without sacrificing performance. Experimental evaluations on edge devices with multiple different processor architectures demonstrate the efficacy of the framework, achieving a model compression rate ranging from 15% to 70% with less than a 1% reduction in accuracy.