Probabilistic Deep Learning for Energy Time Series Forecasting: A Comparative Study

Deep learning methods have achieved widespread success and are continually being utilized in new domains, including the prediction of energy time series. However, these methods do not indicate their confidence in predictions, hindering the ability to evaluate, interpret, and enhance forecast accuracy. As a result, this research delves into the exploration of probabilistic extensions to deep learning and their implementation in energy time series forecasting. The evaluated methods include concrete dropout, deep ensembles, Bayesian neural networks, deep Gaussian processes, and functional neural processes. Two load forecasting scenarios are considered for evaluation: short-term (single-step) and day-ahead (multi-step) forecasting. The methods are assessed based on calibration, sharpness, and proficiency in indicating a lack of knowledge. To provide a reference, a simple neural network and a quantile regression model are also employed. Overall, the methods demonstrate strong performance, with concrete dropout, deep ensembles, and Bayesian neural networks performing equally well or better than the reference methods. Functional neural processes and deep Gaussian processes exhibit weaker performance, likely attributed to convergence issues and suboptimal parameters. Notably, deep ensembles prove to be straightforward to implement and train, requiring very few hyperparameters.