Neural network-based prediction interval estimation with large width penalization for renewable energy forecasting and system applications

Increasing the penetration of renewable energy introduces significant uncertainty into power systems. Probabilistic forecasting, which quantifies this uncertainty through prediction intervals (PIs), is essential for guiding a generation operating reserve preparation. The amount of standby generation resources is directly reflected by a PI width and typically focuses on the worst-case scenario arising with large PI widths under extreme conditions. This paper aims to reduce the large PI widths by proposing a new PI-based loss function that utilizes the sum of the K-largest element functions to impose greater penalties on larger PI widths in developing a renewable energy forecasting model. The proposed methodology can identify and reduce large PI widths during the model training process while ensuring PI’s reliability. The loss function is compatible with gradient-based algorithms, allowing for further integration with state-of-the-art neural networks and recent deep learning techniques. Experiments on synthetic and solar irradiance forecasting datasets utilizing ANN and LSTM models showcase our approach’s effectiveness in attaining narrower PIs while maintaining prediction accuracy. A cost analysis of solar power reserve demonstrates that our method yields reduced reserve over-allocation and lower total costs for provision and deficit penalties under high uncertainty. This is due to an improved PI’s lower bound, which better captures actual generation, thereby decreasing lost load penalties. Furthermore, in robust energy management, the net electricity cost range assessed using PI information exhibits the narrowest variation compared to benchmarked methods due to the conservatism reduction in PI widths of net load forecasts. • A new loss function for prediction interval estimation that aims to achieve a substantial reduction in the magnitude of large interval widths. • A significant reduction in large prediction interval widths shown in solar irradiance forecasting. • Application on reducing conservatism in worst-case robust energy management and reserve preparation cost analysis.