In-field estimation of vertical distribution of total nitrogen and nicotine content for tobacco plants based on multispectral and texture feature fusion

Accurately obtaining the total nitrogen and nicotine content of tobacco plants and their vertical distribution within the canopy is crucial for smart management and quality assessment. However, the complex field environment and uneven vertical distribution pose significant challenges for precise estimation. This study proposed a spectral and texture feature fusion method based on deep learning to improve estimation accuracy, and an improved YOLOv8 model (AO-YOLOv8) was developed for tobacco leaf instance segmentation. After segmentation, the average spectral features from six image channels were extracted, and 474 texture features were obtained using Gray Level Co-occurrence Matrix (GLCM), Local Binary Pattern (LBP), Fourier Transform, Gabor Filter, and Wavelet Transform. Four deep neural networks, including LSTM, RNN, MLP, and FCNN, were then applied to establish estimation models of nitrogen and nicotine content at both the leaf and plant scales. The results showed that AO-YOLOv8 achieved an mAP50 of 87.3 and an mIoU of 83.4 in the leaf instance segmentation task, representing improvements of 6.99% and 8.88% over the original YOLOv8, and effectively detected and separated overlapping leaves under complex conditions. The fusion of spectral and texture features significantly improved prediction accuracy, with the LSTM network achieving the best performance, yielding R² values of 0.8634 and 0.8735 for nitrogen and nicotine prediction at the leaf scale in laboratory conditions. In the field environment, the LSTM-based models for plant-scale nitrogen and nicotine estimation achieved R² values of 0.6771 and 0.5735, respectively, which outperformed models using spectral features alone. In conclusion, this study enabled accurate estimation and visualization of the vertical distribution of nitrogen and nicotine content in field-grown tobacco plants, providing an efficient, low-cost, and non-destructive solution for tobacco production and quality control.