Multispectral AI-driven imaging for detection of downy mildew and gray mold in grapevines

• Dual-head SegFormer enables robust grape disease segmentation • Multispectral and depth data improve disease detection accuracy • YOLO-derived masks boost leaf segmentation IoU by over 11% • 15-channel fusion enhances pixel-level classification performance • Framework supports UAV and robotic precision agriculture systems Downy mildew ( Plasmopara viticola ) and gray mold ( Botrytis cinerea ) are among the most destructive grapevine diseases worldwide, causing substantial yield losses and compromising fruit quality. Traditional diagnostic methods based on visual assessment and microscopic examination are time-consuming, labor-intensive, and require considerable expertise. This study presents a novel computer vision approach for automated grape disease detection by combining instance and semantic segmentation techniques on multispectral imagery. A dataset of 451 captures comprising RGB and five-band multispectral images (460, 540, 640, 780, 880 nm) was collected from open-field vineyards, including healthy, gray mold–symptomatic, and downy mildew–symptomatic leaves. Two complementary approaches were developed: (i) a YOLOv11-based instance segmentation model for rapid leaf identification, and (ii) a dual-head SegFormer architecture for semantic segmentation incorporating 15 input channels, including RGB, multispectral bands, derived vegetation indices, YOLO-generated masks, and depth information. The dual-head SegFormer includes a primary multiclass segmentation head and a secondary binary head for leaf–background discrimination, with consistency regularization between heads to enhance performance. The YOLOv11 model achieved 89.7% mAP50 for leaf segmentation. The SegFormer-based model achieved an overall mean IoU of 75.22%, an F1-Score of 83.53% and a single-class leaf segmentation IoU of 91.79%. Disease-specific segmentation performance was high, with gray mold achieving 94.32% IoU and an F1-score of 97.08%, while downy mildew achieved 75.5% IoU and an F1-score of 86.04%. The integration of multispectral channels and derived indices improved mean IoU by 3–5%, while the inclusion of YOLO-derived masks and depth information from the Depth Anything V2 model increased single-class IoU by more than 11%. The proposed framework demonstrates strong capability for disease-specific detection and is well suited for UAV- and ground robotics–based precision agriculture applications.