Multi-modal learning from image-derived graphical construction to improve ultra-short-term photovoltaic power forecasting

• An image-derived graphical construction method is proposed to extract key features. • A super-pixel segmentation method is developed for image processing. • Various segmentation methods in sky images are sufficiently evaluated. • A competitive photovoltaic power fluctuation forecast result is obtained. Solar energy plays a crucial role in addressing global climate change and energy crisis. However, the high volatility and instability of photovoltaic (PV) power generation, driven by complex cloud motion, pose significant challenges for grid integration. Accurate PV power forecasting with the assistance of sky images is key to managing the variation. Few existing researches have comprehensively explored inter-image coupling features that could capture complex cloud-driven dynamic and enhance forecasting precision. Consequently, this paper proposes a novel ultra-short-term PV power forecasting framework based on multi-modal learning from image-derived graphical construction. Firstly, this research develops super-pixel segmentation to overcome the limitations on capturing complex pixel relationships and enhance boundary-texture recognition. Subsequently, this method proposes a graph structure to extract global coupling features across sky images chunks and historical data, then applies Graph Sample and Aggregate network to learn embedded representations of graph. Finally, a cross-modal feature synergy fusion module based on multi-headed attention mechanism is proposed to explore coupling correlations between different modalities that enable global interaction and redundancy elimination. Experimental results show that the proposed method achieves a normalized root mean square error (nRMSE) of 6.1% in 5-min-ahead forecasting. Compared to eleven mainstream models, the proposed method surpasses them across various performance indicators, particularly under cloudy and rapidly fluctuating conditions. This work expands the research on PV power forecasting and provides vital numerical support for achieving stable and efficient grid integration of PV power generation.