MHRA-MS-3D-ResNet-BiLSTM: A Multi-Head-Residual Attention-Based Multi-Stream Deep Learning Model for Soybean Yield Prediction in the U.S. Using Multi-Source Remote Sensing Data

Accurate prediction of soybean yield is important for safeguarding food security and improving agricultural management. Recent advances have highlighted the effectiveness and ability of Machine Learning (ML) models in analyzing Remote Sensing (RS) data for this purpose. However, most of these models do not fully consider multi-source RS data for prediction, as processing these increases complexity and limits their accuracy and generalizability. In this study, we propose the Multi-Residual Attention-Based Multi-Stream 3D-ResNet-BiLSTM (MHRA-MS-3D-ResNet-BiLSTM) model, designed to integrate various RS data types, including Sentinel-1/2 imagery, Daymet climate data, and soil grid information, for improved county-level U.S. soybean yield prediction. Our model employs a multi-stream architecture to process diverse data types concurrently, capturing complex spatio-temporal features effectively. The 3D-ResNet component utilizes 3D convolutions and residual connections for pattern recognition, complemented by Bidirectional Long Short-Term Memory (BiLSTM) for enhanced long-term dependency learning by processing data arrangements in forward and backward directions. An attention mechanism further refines the model’s focus by dynamically weighting the significance of different input features for efficient yield prediction. We trained the MHRA-MS-3D-ResNet-BiLSTM model using multi-source RS datasets from 2019 and 2020 and evaluated its performance with U.S. soybean yield data for 2021 and 2022. The results demonstrated the model’s robustness and adaptability to unseen data, achieving an R2 of 0.82 and a Mean Absolute Percentage Error (MAPE) of 9% in 2021, and an R2 of 0.72 and MAPE of 12% in 2022. This performance surpassed some of the state-of-the-art models like 3D-ResNet-BiLSTM and MS-3D-ResNet-BiLSTM, and other traditional ML methods like Random Forest (RF), XGBoost, and LightGBM. These findings highlight the methodology’s capability to handle multiple RS data types and its role in improving yield predictions, which can be helpful for sustainable agriculture.