Alpine-Peaks Shape of Optimized Configurations Post Random Search in Regional Hydrological LSTMs

In hydrological prediction, Long Short-Term Memory networks (LSTMs) hold promise, yet precise hyperparameter tuning remains challenging, especially for regional prediction across multiple catchments. This study addresses the challenge by employing a random search approach to simultaneously tune 10 distinct hyperparameters with two learning rate schedules, for hourly streamflow and water level predictions across 40 catchments in Basque Country, Spain. Exploration of 1000 random configurations revealed an optimization landscape akin to Alpine peaks in the hyperparameters space, with promising settings emerging, each showing high accuracy in several basins, but not everywhere. Post-random search analysis identified top-performing configurations exhibiting consistency in regional metrics but variability in basin-scale performance. Underwent the same training approach, some configurations focused to learn general patterns in as many places as possible; while, some tried to learn anomalies in data such as human fingerprints. Therefore, selecting the optimal regional configuration proved complex due to variations in catchment-scale performance. The Alpine-peaks shape of hyperparameter optimization highlights different learning behaviors by different configurations, emphasizing the importance of ensemble learning for improved predictions across diverse catchments.