Short-Term Photovoltaic Power Forecasting Using PV Data and Sky Images in an Auto Cross Modal Correlation Attention Multimodal Framework

The accurate prediction of photovoltaic (PV) power generation is crucial for improving virtual power plant (VPP) efficiency and power system stability. However, short-term PV power forecasting remains highly challenging due to the significant impact of weather changes, especially the complexity of cloud motion. To this end, this paper proposes an end-to-end innovative deep learning framework for data fusion based on multimodal learning, which utilizes a new auto cross modal correlation attention (ACMCA) mechanism designed in this paper for feature extraction and fusion by combining historical PV power generation time-series data and sky image data, thereby enhancing the model’s prediction performance under complex weather conditions. In this paper, the effectiveness of the proposed model was verified through a large number of experiments, and the experimental results showed that the model’s forecast skill (FS) reached 24.2% under all weather conditions 15 min in advance, and 24.32% under cloudy conditions with the largest fluctuations. This paper also compared the model with a variety of existing unimodal and multimodal models, respectively. The experimental results showed that the model in this paper outperformed other benchmark methods in all indices under different weather conditions, demonstrating stronger adaptability and robustness.