Multimodal sequential cross-modal transformer for predicting plant available water capacity (PAWC) from time series of weather and crop biological data

Deep learning (DL) and machine learning (ML) have been applied widely to satellite data of vegetation indices to infer indirect features associated with soil characteristics that affect crop performance in rain-fed environments. In this paper, we propose a DL model for prediction of plant available water capacity (PAWC) of the soil from sequential multi-modal data including time series of biomass, leaf area index (LAI), normalised difference vegetation index (NDVI), and cumulative weather variables. By initiating large numbers of simulations with different soil PAWC, weather and management parameters, we explore combinations of the simulation outputs and the weather to estimate the PAWC and to determine the factors that impede the accuracy of the prediction model. Experimental results demonstrate the significant potential of our method compared with traditional ML methods. Specifically, our method increases the prediction accuracy in situations where each PAWC profile is grouped into two or five classes of PAWC. For more classes (10 classes), the model achieves more than 60% for the overall accuracy and performs well on the lowest five PAWC classes. The utilisation of sequential multi-modal data to predict soil water level provides a direction for future work to translate onto empirical datasets and also to explore the boundaries of the prediction ability of DL models.