Enhancing drought prediction through machine learning: advanced techniques combining phenotypic and agrometeorological data

• The multimodal approach improves drought classification accuracy by 5.75 % over single-modality methods. • Mutual Information (MI) based feature selection reduces dimensionality and improves computational efficiency. • Handcrafted features outperform CNN features in scenarios with limited training samples. • GOA optimization boosts the performance of RF and XGBoost models for accurate predictions. Accurate monitoring of drought is crucial as it severely impacts normal plant growth and presents significant challenges to sustainable agriculture and global food security. However, as in most studies, drought monitoring that relies on unimodal data may not effectively capture the intricate nature of drought phenomena. This study proposes a multimodal framework that combines drought-stressed winter wheat images with field-collected agrometeorological data to develop robust machine-learning models. Two machine learning algorithms, Random Forest (RF) and Extreme Gradient Boosting (XGBoost), were evaluated as predictive models. Image-based phenotypic features were extracted using a CNN-based network, resulting in 512 features, while handcrafted methods generated 48 features. To enhance efficiency, mutual information (MI) was utilized for feature selection, and the grasshopper optimization algorithm (GOA) was employed for hyperparameter tuning. The experimental results showed that the proposed GOA-optimized RF model achieved an average accuracy of 98.75 %, outperforming the unimodal model by 5.75 %. Furthermore, our findings demonstrate the effectiveness and superior performance of the proposed method under various challenging data conditions, such as noise and occlusion, which hold great potential for real-world agricultural applications.