A bilevel-optimized multimodal transformer framework for plant disease classification and severity prediction using adaptive feature fusion

Early and accurate identification of plant diseases is important for improving crop productivity and supporting sustainable agricultural practices. However, conventional vision-based approaches often fail to utilize complementary environmental information that may be associated with plant health and disease-conducive conditions. This study proposes a bilevel-optimized multimodal transformer framework for plant disease classification and severity prediction using adaptive feature fusion. The proposed framework integrates visual leaf images with complementary environmental information through a unified architecture comprising a Dual-Context Hierarchical Transformer for feature extraction, an adaptive multimodal fusion mechanism, multi-task learning for joint disease classification and severity estimation, and an uncertainty-aware prediction module based on Monte Carlo dropout. A Modified Bilevel Unified Optimization strategy coupled with the Marine Predators Algorithm (MBUO-MPA) is employed to jointly optimize model parameters and hyperparameters. Since the publicly available visual and environmental datasets used in this study were independently collected and do not contain naturally paired observations, a synthetic alignment strategy is adopted to investigate heterogeneous multimodal feature integration. The proposed framework is evaluated using the New Plant Diseases Dataset, 20 K Crop Disease Dataset, Wheat Plant Diseases Dataset, and Crop Recommendation Environmental Dataset. Experimental results demonstrate a maximum classification accuracy of 98.74%, together with strong precision, recall, F1-score, and AUC values. Ablation and cross-dataset analyses further demonstrate the contribution of the proposed architectural components under the evaluated benchmark conditions. The reported multimodal results represent a methodological evaluation using synthetically aligned heterogeneous datasets and should not be interpreted as validation using naturally synchronized field observations. Future work will focus on validation using synchronized plant imagery, environmental sensor measurements, and expert-verified disease observations collected under operational agricultural conditions.