A multimodal fusion framework for prediction of hotspot-induced photovoltaic performance loss

Hot spot effects can cause rapid power loss and local temperature rise in photovoltaic (PV) modules, posing significant risks to system efficiency and operational safety. Accurate prediction of hotspot-induced power loss and temperature rise is important for effective monitoring and preventive maintenance. However, existing methods often rely on a single numerical modality, which limits the ability to capture spatial information such as hot spot size and consequently restricts prediction accuracy. To address these limitations, this study proposes a multimodal fusion–based prediction framework under controlled experimental conditions, termed MMF-HSPP, for the short-term hotspot-induced PV power loss and temperature rise prediction. The proposed framework integrates visual images, environmental and electrical parameters, and thermal infrared information to achieve collaborative spatiotemporal and thermal perception. A fine-tuned ResNet18 model with a focal loss mechanism is developed to accurately quantify hot spot occlusion areas, particularly for small-scale hot spots. In addition, an uncertainty-aware multimodal fusion strategy combining uncertainty weight optimization and dynamic weight adjustment is introduced to suppress cross-modal error propagation and improve fusion robustness. Experimental results demonstrate that the proposed method achieves a mean relative error of 2.45% in hot spot occlusion area prediction. For power loss rate prediction, an R 2 of 0.9484 with NRMSE of 4.53% is obtained. For temperature rise prediction, the R 2 reaches 0.8374 with NRMSE of 8.10%. These results demonstrate the effectiveness of the proposed multimodal fusion framework in enabling robust data-driven modeling of the relationship between hotspot spatial characteristics and photovoltaic performance loss under controlled conditions.