Interpretable Machine Learning for Multi-Crop Yield Prediction in Semi-Arid Regions: A Hierarchical Approach to Handle Climate Data Sparsity

This study develops a hierarchical machine learning framework to address the challenges of multi-crop yield prediction in semi-arid regions, focusing on sparse climate data, model interpretability, and heterogeneous climate-crop interactions. The framework integrates Random Forest (RF) techniques for crop-specific modeling, exploiting their robustness to nonlinear relationships in small datasets, while long short term memory (LSTM) networks extract long-term climate trends, and Convolutional Neural Network (TCNN) with dilated kernels capture multi-scale temporal dependencies, creating a complementary hierarchy that leverages the strengths of each approach. Missing weather data (gaps of 8.3–45.2%) are addressed via Multiple Imputation by Chained Equations (MICE), incorporating climatological constraints to preserve physical consistency. Model interpretability is achieved through SHapley Additive exPlanations (SHAP) analysis and uncertainty decomposition, quantifying the contributions of data variability, temporal dynamics, and model ensembles. Validated on 13 years of agricultural data (2010-2022) from the semi-arid Ouarzazate region of Morocco, the framework achieved R² values of 0.77 (cereals), 0.73 (vegetables), and 0.70 (tree crops), representing a 12-18% improvement over conventional single-model approaches which typically achieve R² values of 0.65-0.68 in similar semi-arid conditions with complete datasets. SHAP analysis identified crop-specific critical thresholds, including maximum summer temperatures (>40°C) for vegetables and winter precipitation (<30mm) for cereals, while uncertainty quantification revealed that transition seasons (spring/autumn) are high-risk periods with prediction variances exceeding ±1.5 T/ha. Performance declines during extreme weather events (e.g., unprecedented droughts) that are absent from historical data, and soil moisture dynamics remain excluded. Future work will address these limitations through integration of soil moisture sensors, development of transfer learning approaches for extreme weather events. Practical applications include optimizing irrigation schedules using SHAP-derived temperature thresholds and prioritizing resource allocation during periods of high uncertainty. This work represents a novel unified framework combining imputation, temporal modeling, and crop-specific interpretability for semi-arid systems, going beyond black-box models or single-crop approaches. Keywords: Hierarchical machine learning, agricultural yield prediction, multi-crop systems, shap interpretability, missing meteorological data.