Fast detection of rice striped stem borer (Chilo suppressalis) stress based on UAV sensor and multimodal segmentation method

The rice stem borer (Chilo suppressalis, SSB) is a major pest threatening rice yields. While unmanned aerial vehicle (UAV) technology has demonstrated potential in crop SSB-stress monitoring, current methodologies predominantly rely on single-modal optical data, particularly RGB images, thereby limiting the comprehensive assessment of the multidimensional phenotypic alterations caused by SSB infestations. To address this limitation, we developed a novel Multimodal Infestation Segmentation Network (MISeg-Net) that incorporates a dual-branch architecture to synergistically integrate RGB optical features and digital surface models (DSM) structural features. The network’s innovative design combines Dual Atrous Spatial Pyramid Pooling (DASPP) with a multi-perspective attention mechanism, facilitating multi-scale contextual feature extraction and adaptive cross-modal feature optimization. This advanced architecture significantly enhances the recognition accuracy of pest-infected regions within complex canopy environments, effectively overcoming the constraints of traditional algorithms in terms of multimodal feature fusion efficiency and recognition accuracy. Experimental evaluations demonstrate that the superior performance of MISeg-Net over conventional single-modal models (e.g., U-Net, PSPNet), with mIoU of 91.77%, PAcc of 97.48%, and F1-score of 98.66%. Furthermore, the pest-infested area ratio derived from segmentation exhibits a strong correlation (R² = 0.859) with manually measured dead heart rates. This research provides an efficient solution for precision monitoring of SSB infestations using UAV-based technology.