Empirical Evaluation on GPU, Overclocking, and LoRA for Deep Learning on Embedded Systems

The use and optimization of deep learning models for embedded systems and mobile devices present real-world challenges, such as limited computational resources, memory size, inference latency, and high-power consumption requirements. This study evaluated 14 models from nine popular deep learning architectures using three hardware platforms (Jetson Nano B01, Jetson AGX Xavier, and GTX 1650 laptop). The trade-offs between reducing training time, image throughput, accuracy loss, temperature increase, and power consumption were analyzed using two strategies: 1) overclocking and 2) low-rank adaptation (LoRA). The results in the overclocking mode show a reduction in the training time from 3.7% (GoogLeNet) to 38.9% (EfficientNet-B0), and the temperatures have an average increase of 0.5-21.9°C when considering the CPU, GPU, and all devices. Our accuracy results for all devices were superior to those of the PyTorch baseline on ImageNet and aligned with the performance of each device. Regarding LoRA, we observed an average reduction in training time of 70% in Jetson Nano, but an average accuracy loss ranging from 21% to 40% in all models. What distinguishes our study is its multi-platform, multi-strategy evaluation, which combines hardware tuning with software training strategies. The findings of this study will contribute to other researchers in discerning and choosing algorithms to be used in the deployment and retraining of deep learning models on embedded and mobile platforms that have not yet been well explored in the literature.