Diagnosis of early nitrogen, phosphorus and potassium deficiency categories in rice based on multimodal integration and knowledge distillation

Rapid, non-destructive, lightweight and accurate diagnosis of early stage nutrient deficiency in rice is essential for both yield and quality. Traditional diagnostic methods often exhibit low efficiency, reduced accuracy, and a lack of timeliness. To address these issues, a diagnostic method for the early detection of nitrogen, phosphorus, and potassium deficiencies in rice, based on multimodal integration and knowledge distillation, is proposed. In this study, the late rice variety 'Huanghuazhan rice' was selected as the experimental subject for field trials. First, leave images of rice plant were captured using a scanner, and some data preprocessing techniques were utilized to extract image samples from the leaf tip areas of the top one leaf, the top two leaf and the top three leaf. Second, the teacher model was obtained through transfer learning, fine-tuning training and model fusion. The custom neural network model was heuristically customized based on the conventional model. The teacher model then performs knowledge distillation on the custom neural network model, resulting in a lightweight model with high accuracy and low memory consumption, which serves as a feature extractor. Finally, the multimodal features were input into LightGBM for training and the rice nutrient deficiency recognition model, S-RiceNet-D-LightGBM (SRDL), was constructed. The experimental results demonstrate that the SRDL model is an efficient, lightweight model characterized by high accuracy and low memory consumption. It achieved an accuracy score of 0.9501, a macro precision score of 0.9501, a macro recall score of 0.9499, and a macro F1 score of 0.9500, outperforming the VGG16, ResNet101, DenseNet169, InceptionNetV3, MobileNetV2, second only to the performance of the ensemble model. The memory footprint is 23.6 MB, which is slightly higher than that of the MobileNetV3S model. This study provides new insights and viable avenues for the practical implementation of a lightweight model designed for the intelligent diagnosis of crop nutrient deficiency.