Semantic Cloud Segmentation Models for Solar Applications with Synthetic Data

Highly resolved intra-hour solar irradiance forecasts benefit the solar industry by boosting plant efficiency, storage integration, energy trading, and grid stability. These forecasts typically rely on ground-based all-sky imagers (ASIs) and irradiance measurements. Semantic cloud segmentation is key for physics-based and certain data-driven ASI forecasting methods. Despite advances in computer vision, semantically segmenting sky images into distinct cloud classes remains challenging. Similar visual traits and unclear boundaries in certain cloud types complicate segmentation, especially in overlapping multi-layer scenarios. Distinguishing thin high-altitude clouds from atmospheric turbidity and aerosols is difficult due to fuzzy edges. Variability in cloud appearance, caused by shifting light, atmospheric effects, and fish-eye lens distortion, adds further complexity. Recent studies [1] classify clouds into low-, mid-, and highlayer categories, plus clear sky, based on WMO definitions [2]. Though simplified, this approach highlights how cloud composition affects solar irradiance. High-layer clouds (e.g., Cirrostratus, Cirrocumulus), composed of ice, typically reduce irradiance slightly. Dense lowlayer clouds (e.g., Cumulus, Stratus), primarily water, dim it significantly. Mid-layer clouds (e.g., Altocumulus, Altostratus), blending traits of both, have a broad variety. Individual cloud layers show distinct dynamics such as movement, speed, development, or dissipation, driven by tropospheric wind and atmospheric conditions. Reliable deep learning models for semantic cloud segmentation require extensive, highquality pixel-level annotations. However, manual annotation is costly and impractical for large datasets. Recent automated methods blending self-supervised and weakly-supervised techniques with limited manual ground truth data show promise [1, 3]. This work advances automated annotation by exploiting temporal relationships in sequential images through semi-supervised video object segmentation, leveraging distinct cloud layer dynamics. The application of video object segmentation to clouds has not been explored yet highlighting the novelty of this approach. [1]: https://doi.org/10.5194/amt-15-797-2022 [2]: https://cloudatlas.wmo.int/en/home.html [3]: https://elib.dlr.de/204657/