Multimodal ultra-short-term probabilistic solar power forecasting with generative AI and Transformer

Solar power is one of the most widely adopted forms of renewable energy, with its usage rapidly increasing in recent years. However, solar power generation is highly sensitive to environmental factors, leading to frequent fluctuations. These fluctuations present challenges for large-scale integration into the power grid. Sky images, which provide real-time information about sky conditions, can help improve ultra-short-term solar power forecasting. Nonetheless, due to the inherent uncertainty of cloud movement, solar power generation can be unstable over short time intervals. This instability calls for the inclusion of uncertainty in ultra-short-term forecasting models. Inspired by recent advances in deep learning, we propose a novel framework for ultra-short-term probabilistic solar power generation forecasting. This framework uses both historical solar power generation data and sky images as inputs. First, a video prediction model based on generative artificial intelligence (AI) is applied to generate multiple potential future sky image sequences. Next, a Transformer-based multimodal model combines each predicted sequence with historical solar power generation data to derive a distribution of possible future power outputs. To account for uncertainties in both the video prediction and multimodal models, these distributions are aggregated into a single overall distribution. We conducted experiments on real-world datasets, with prediction times ranging from 15 to 60 min, to compare our method with previous mainstream approaches. The results demonstrate that our method outperforms existing approaches in both deterministic and probabilistic forecasting tasks. In the 15 min ahead forecasting task, compared to the method using only historical solar power generation data, our method reduces the Root Mean Square Error (RMSE) of the deterministic evaluation metric by 20.6% and the Continuous Ranked Probability Score (CRPS) of the probabilistic evaluation metric by 19.4%. When compared to the method using only sky image data, our approach reduces the RMSE by 47.3% and the CRPS by 51.3%. Furthermore, we conduct additional analysis on the performance of various methods under different weather conditions.