Optimized FLOPs-Aware Knowledge Distillation for TinyML Applications in Agriculture

This study introduces a novel FLOPs-Aware Knowledge Distillation (FAKD) framework tailored for TinyML applications in agriculture. By integrating Floating Point Operations (FLOPs) as a regularization term into the distillation loss, the framework ensures computational efficiency while preserving accuracy. The teacher-student architecture employs ResNet-50 and MobileNetV2, respectively, with the latter optimized for edge deployment. The methodology includes advanced techniques such as pruning and INT4 quantization, enabling the model to fit stringent hardware constraints, such as the ESP32's 1MB PSRAM, while maintaining high test accuracy. Experimental results on the PlantVillage dataset reveal a significant accuracy improvement from 92.77% to 96.55% post-FAKD, followed by a tradeoff to 90.15% after optimization for deployment. Furthermore, the study compares the performance of the optimized model across different devices, demonstrating a balance between accuracy, inference time, and power consumption. The results validate the framework's potential for deploying efficient and accurate models on low-power devices in resource-constrained environments, making it suitable for smart agriculture applications. This work underscores the importance of balancing model performance and computational efficiency, paving the way for sustainable TinyML solutions in agriculture.