Identification method for multiple similar crop pests: A visual language model combined with coarse–finegrain cross‐modal matching

BACKGROUND: Crop pests and diseases stand as one of the paramount issues within the agricultural domain. However, the resemblance in pest characteristics and background information within real field environments poses substantial challenges to the identification of crop pests. Additionally, the similarity among pests themselves further complicates this process. Conventional approaches reliant solely on single-modal feature extraction find it increasingly arduous to contend with such intricately dynamic surroundings. RESULTS: Thus, this study introduces a two-phase cross-modal model for crop pest identification. In the first phase, a cross-modal matching approach at both coarse- and fine-grained scales is employed to acquire crucial features of pests. This method performs cross-modal matching separately on image-text pairs and finely partitioned segments of image-text features, which captures key feature segments that facilitate identification. In the second phase, a 'point-to-area' key feature super-localization module is devised. By comparing the proximity of obtained key feature segments in the preceding step through similarity, these features are further localized to obtain complete key feature regions. These regions are then embedded back into the feature map to enhance the network's capability in localizing key feature regions. CONCLUSION: Our approach is validated using the IP102 dataset collected from real field environments and the dataset provided by Li and Xie. Compared to previous cross-modal models, the classification average accuracy improves by 11.68%, 4.26% and 5.90%, respectively. Experimental results demonstrate that the model is suitable for monitoring and detecting crop pests and diseases in complex environments. © 2026 Society of Chemical Industry.