ConvODE-Mixer: A multimodal deep learning model for ultra-short-term PV power forecasting

Solar energy has emerged as a critical renewable resource for addressing global energy and environmental challenges. Owing to meteorological-induced stochastic fluctuations in photovoltaic (PV) generation, PV power forecasting still faces significant challenges, potentially causing grid instability events. This paper proposes a multimodal model, designated ConvODE-Mixer, integrating convolutional neural networks (CNNs) with neural ordinary differential equations (NODE) to improve the ultra-short-term PV power forecasting accuracy. By integrating ground-based cloud images (GBCI) and meteorological data, ConvODE-Mixer utilizes a multi-scale lite-reduced atrous spatial pyramid pooling (LR-ASPP) segmentation module to capture cloud thickness variations and a channel attention mechanism that dynamically weights light transmittance-sensitive features, thereby enhancing PV power forecasting precision. In the 10 min ahead forecasting task, ConvODE-Mixer exhibited statistically significant performance enhancements over MNF-ODEnet. Specifically, ConvODE-Mixer achieved a 40.45% reduction in mean square error (MSE), a 31.11% decrease in mean absolute error (MAE), a 4.66% improvement in R 2 , and a 41.17% reduction in relative absolute error (RAE). These results validate the model’s capacity to stabilize ultra-short-term grid operations by reducing prediction-to-actual deviations during rapid weather transitions, thereby enabling power dispatch systems to maintain supply–demand equilibrium with improved operational efficiency. • ConvODE-Mixer integrates CNNs and Neural ODEs for 10 min photovoltaic forecasting. • LR-ASPP semantic segmentation enhances ground-based cloud imaging accuracy. • Reduces MSE by 40.45% and MAE by 31.11% versus MNF-ODEnet. • Fuses cloud images and meteorological data for multimodal PV forecasting.