Comparative Benchmark of Metaheuristic and Model-Based Hyperparameter Optimization for Deep Learning Models in Global Solar Irradiance Forecasting

Deep learning solar forecasters depend strongly on their hyperparameters, yet comparisons of hyperparameter optimization methods usually rely on few runs per optimizer, confounding the optimizer with the random seed. A protocol is proposed that tests whether optimizers are practically equivalent and how reliable published comparison designs are. Seven methods (four metaheuristics, two model-based methods, one of them multi-fidelity, and Random Search) tuned three deep networks for one-hour-ahead forecasting of global horizontal irradiance in Querétaro, Mexico, with public reanalysis data, a shared search space, one hundred evaluations, and thirty paired seeds, in nine hundred and thirty runs across all experiments. The tuned models improved on smart persistence by 4–11% in mean absolute error. With the 10-epoch search objective, no pair of optimizers differed after Holm correction, 62 of 63 pairs were equivalent within 1 W/m2, and the optimizer explained 0.8% of the error variance, compared with 7.9% for the seed. In a higher-fidelity check with 50 epochs, two guided methods improved on Random Search by 0.74 W/m2, significantly for one of them, while remaining equivalent to each other, suggesting that, in this setting, objective fidelity mattered more than the choice among guided optimizers. In emulations of published designs, even with 30 runs, 2 studies agreed on the winner in at most 55% of cases. A validated surrogate benchmark, code, data, and predictions are released.