Efficient training of deep neural networks on resource-constrained devices for industrial applications

Deep neural networks (DNNs) have shown remarkable success across many artificial intelligence applications, yet their computational and memory demands make them difficult to deploy on low-resource devices such as Raspberry Pi, microcontrollers, and mobile edge hardware. In real-time scenarios such as remote health monitoring, smart agriculture, and IoT-based automation create barriers to on-device AI implementation. This study focuses on optimizing DNNs for resource-constrained environments by employing Python-based techniques to compress and accelerate models without substantial accuracy loss. We propose a three-stage optimization approach combining pruning, quantization, and knowledge distillation. Model pruning reduces the number of unnecessary parameters, quantization converts model weights to low-precision formats (e.g., int8), and knowledge distillation transfers knowledge from a large teacher model to a smaller student model. These techniques are implemented using TensorFlow Lite and PyTorch Mobile frameworks.