Lightweight Deep Learning for Resource-Constrained Devices

With the increasing usage of deep learning models on mobile and resource-constrained hardware, efficient model design is necessary so that accuracy comes at the same cost as resource utilization. Several techniques for producing lightweight deep learning models for real-time inference on mobile hardware are reviewed in this paper. Three fundamental methods are pruned, quantized, and optimized; each reduces the complexity of the model but does not detract from the performance. The practical issues and trade-offs of applying these methods on devices with limited memory, power, and computational resources are analyzed. By concrete case studies and benchmark results, the ways in which these methods enable deep learning in real-world mobile applications, such as computer vision, natural language processing, and augmented reality, are recorded. The paper concludes with the discussion of emerging trends in lightweight model development and future research directions that may have a relation with these optimized models, driving innovations in mobile AI and edge computing.