HOG-GRU: a novel multi-modal water level forecasting model integrating satellite imagery and reservoir operation data

Purpose Forecasting reservoir water levels plays a critical role in effective water resource management, contributing to the safety of hydraulic infrastructure and mitigating the impacts of droughts and floods. Current forecasting models work solely on satellite imagery, which has limitations on handling noise, particularly in peak values within the time-series streamflow from reservoir operation. Indeed, a multi-modal approach that integrates both satellite imagery and reservoir operation data is necessary to enhance the performance of forecasting. Design/methodology/approach This research presents a novel multi-modal forecasting model that integrates satellite imagery with historical water level data to improve prediction accuracy, particularly in forecasting abrupt changes in water levels. Image features are extracted using the histogram of oriented gradients (HOG) algorithm and normalized with the L2 norm to enhance training stability and reduce noise. A customized fusion function is developed to combine spatial features from satellite imagery with temporal features from water level time series, resulting in a unified composite feature vector. This vector, along with the historical water level sequence, is fed into a gated recurrent unit (GRU) model for forecasting. The fusion mechanism plays a crucial role in capturing sudden and abnormal variations in the data. Findings The model is assessed using satellite images and on-site water level measurements collected at the An Khe and Ka Nak Reservoir, Gia Lai, Vietnam, spanning January 2019 to December 2022. Experimental results demonstrate that the HOG-GRU variant significantly outperforms conventional deep learning models. The specific evaluation metrics are as follows: for the An Khe Reservoir, mean squared error (MSE) (0.08060), root mean squared error (RMSE) (0.28390), mean absolute error (MAE) (0.20446) and |Tracking Signal| (0.00032); whereas for the Ka Nak Reservoir, the corresponding values are MSE (0.20795), RMSE (0.45601), MAE (0.37937) and |Tracking Signal| (0.03985). These findings confirm the model's robustness and its practical applicability to real-world hydrological forecasting tasks. Originality/value This paper presents an original research contribution, offering novel insights to the academic domain of information technology, with all references comprehensively and accurately cited.