Benchmarking Deep Learning Against Statistical Baselines and a Physical Climate-Model Comparator for Station-Scale Meteorological Forecasting: A 100-Station Study from the Western Balkans

Benchmarking deep learning forecasters against classical and physically based numerical baselines remains uncommon in the time-series forecasting literature. Meteorological station networks offer an under-exploited evaluation environment, uniquely providing a physically based climate-model comparator alongside standard baselines. We evaluated eight forecasting approaches—climatology, SARIMA, Random Forest, and five deep learning architectures (TFT, N-HiTS, PatchTST, TiDE, xLSTM)—against bias-corrected output from a five-member CMIP6 ensemble, on 100 meteorological stations across four Western Balkan countries (monthly temperature and precipitation, 1961–2020), using non-parametric significance testing, a rolling-origin backtest (five windows, 2011–2020), and a five-seed robustness check. For temperature, all five deep learning architectures achieved lower MAE than the classical baselines (p < 10−99), though PatchTST’s advantage over climatology was not significant; the best-performing architecture varied across seeds and evaluation windows, so we characterise a leading cluster (N-HiTS, TFT, TiDE, PatchTST) rather than a single winner. The primary temperature advantage was geographically broad-based, while the comparison against the physical-model baseline was robust to the choice of comparator GCM. For precipitation, by contrast, a simple climatological-mean baseline outperformed all five deep learning architectures with no exception across all five rolling-origin windows. The deep learning advantage over classical and physical baselines is thus variable-specific rather than universal. Meteorological station networks, combined with a physically based climate-model comparator, constitute a well-suited evaluation environment for the broader time series forecasting community.