SolarFormer++: Multi-scale Transformer for Solar PV Profiling and Obstruction Localization for Degradation Mitigation

As climate change accelerates, the global transition to clean and sustainable energy is increasingly urgent. Photovoltaic (PV) energy is a preferred choice due to its reliability and ease of installation. However, effective management of PV systems requires (i) precise and real-time localization of solar PV installations for global profiling and (ii) accurate assessments of real-time output voltage and system performance. To address these challenges, we propose a dual-approach solution for comprehensive solar panel analysis leveraging both satellite and UAV imagery. Our method first segments and maps solar panels from satellite images, providing critical insights into their geographic distribution and size. Simultaneously, it analyzes unmanned aerial vehicle (UAV) imagery to prevent degradation caused by obstructions on the solar panel surface, such as leaves, branches, dust, and bird droppings, while also assessing the overall condition of solar PV systems. To further advance the field, we introduce the De-Solar dataset, a novel, high-quality collection of labeled solar panel images focused on diverse obstructions types, designed to train computer vision models. Solar panel identification presents unique challenges due to variations in weather conditions, roof textures, obstruction types, and Ground Sampling Distance (GSD). To address these complexities, we propose SolarFormer++, a state-of-the-art model featuring a multi-scale Transformer encoder and a masked-attention Transformer decoder. The proposed model leverages low-level image features and employs an instance query mechanism to enhance localization of solar PV installations and precise obstruction detection. For global-scale solar PV profiling, we evaluate SolarFormer++ across diverse public datasets, including GGE (France), IGN (France), and USGS (California, USA), spanning varying GSDs for global-scale analysis. To detect obstructions in order to prevent degradation and assess performance, we benchmark the proposed SolarFormer++ on our newly introduced De-Solar dataset. Experimental results consistently demonstrate that SolarFormer++ outperforms existing methods. The source code along with the pre-trained models and dataset, will be made publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/UARK-AICV/SolarFormerPlusPlus</uri>.