Probabilistic forecasting of power system imbalance using neural network-based ensembles

Keeping the balance between electricity generation and consumption is becoming increasingly challenging and costly due to the growing integration of renewables, electric vehicles, heat pumps, and the electrification of industrial processes. Accurate imbalance forecasts, along with reliable uncertainty estimations, enable transmission system operators (TSOs) to dispatch appropriate reserve volumes, reducing balancing costs. Further, market parties can use these probabilistic forecasts to design strategies that leverage asset flexibility for grid balancing, generating revenue with known risks. Despite its importance, system imbalance (SI) forecasting has not been widely studied in the literature. Further, existing methods do not focus on situations with high imbalance magnitudes, which are crucial to forecast accurately for both TSOs and market parties. Hence, we propose an ensemble of constant variable selection networks (C-VSNs), which are a novel adaptation of VSNs. Each minute, our model predicts the imbalance of the current and upcoming two quarter-hours, along with uncertainty estimations for these forecasts. We evaluate our approach by forecasting the imbalance of Belgium, where high imbalance magnitude is defined as | SI | > 500 MW (occurs 1.3 % of the time in Belgium). Results show that, compared to the state-of-the-art, the proposed C-VSN model improves probabilistic forecast performance by 23.4 % in high imbalance magnitude situations and 6.5 % overall, as measured by the continuous ranked probability score (CRPS). Similar improvements are observed for root-mean-squared error (RMSE). Additionally, we introduce a novel fine-tuning methodology that effectively integrates new inputs with limited historical data. This work was performed in collaboration with the Belgian TSO Elia to further improve their imbalance forecasts, demonstrating the relevance of our work. • An ensemble of modified variable selection networks is proposed for probabilistic forecasting of grid imbalances. • We are first to focus on imbalance spikes, which are particularly important for both TSOs and market parties. • Our model strongly outperforms existing methods, especially for imbalance spikes. • The importance of each input and model component is determined via ablation studies. • New features with limited historical data are exploited via fine-tuning.