A multi-modal deep clustering method for day-ahead solar irradiance forecasting using ground-based cloud imagery and time series data

The variability in real-world weather scenarios poses challenges for accurately forecasting solar irradiance. Previous approaches have utilized traditional clustering methods based on historical irradiance series to characterize weather conditions. However, it often overlooks additional valuable information available from cloud imagery and numerical weather prediction (NWP) forecasts. Meanwhile, traditional clustering methods often fail to integrate feature learning and cluster assignment in a mutually reinforcing manner, resulting in sub-optimal clustering performance. Thus, a novel multi-modal deep clustering method is proposed for day-ahead global horizontal irradiance (GHI) forecasting. First, multi-modal deep clustering (MMDC) is employed to categorize samples with similar weather patterns into corresponding clusters. Then, samples from each cluster are used to train multiple multi-modal irradiance forecasting (MMIF) models suitable for different weather conditions. Multi-modal fusion modules are designed to fully learn joint feature contained in multi-modal data, thereby enhancing the accuracy of clustering and forecasting. Experimental results indicate that MMDC-MMIF achieves the lowest root mean squared error (RMSE) of 29.79 W / m 2 . The impact of using different data sources is also analyzed, which shows that fully utilizing multi-modal data for clustering and forecasting can enhance forecasting accuracy and weather robustness. This study is significant for intelligent optimization and management of energy systems. • Propose a novel multi-modal hybrid framework for day-ahead irradiance forecasting. • Fully leveraging features from multi-modal data for both clustering and forecasting. • Mutually reinforcing feature learning and clustering assignment. • The proposed method shows superior forecasting accuracy and weather robustness.