Referring Solar Cell Defect Segmentation in Electroluminescence Images

In the photovoltaic (PV) power generation field, accurately identifying solar cell defects based electroluminescence (EL) images is essential for maintaining high efficiency for PV power plants. Current solar cell defect segmentation methods typically segment all defects in the EL image uniformly, making it difficult to precisely identify specific defects according to maintenance needs. This limitation hinders personalized defect detection for smart operation and maintenance of PV power plants. To solve this problem, a novel task referred to as referring solar cell defect segmentation (RSCDS) is proposed in this article. The goal of the RSCDS task is to precisely segment the specified solar cell defects based on the referring text, tailored to the personalized maintenance requirements of actual PV power plants. Given the lack of relevant datasets, an RSCDS dataset is developed, abbreviated as Ref-EL-defect, comprising 60 000 pairs of defects and corresponding referring texts. The referring text can indicate single defect, multiple defects, or even no defects at all in the EL image, and such multigranularity correspondence enables accurate and personalized segmentation of defects. In addition, a multimodal multigranularity segment network is designed for the RSCDS task. By exploiting the characteristics of the solar cell defects, the multimodal fusion module and multigranularity perception grouping module are proposed to better adapt to the RSCDS task. State-of-the-art (SOTA) referring expression segmentation models designed for natural scene images are transferred to the RSCDS task, and experimental results demonstrate that the proposed method outperforms the SOTA models.