SRP-CR: Semantic and Representational Priors for Diffusion-Based Cloud Removal

The presence of cloud layers severely affects the quality of optical satellite images, greatly limiting the utilization and exploration of subsequent scenes. Recently, state-of-the-art generative models known as diffusion models have been applied to cloud removal in remote sensing (RS) imagery, demonstrating impressive performance. However, current diffusion model-based cloud removal approaches struggle to fully capture priors from degraded multi-temporal RS images due to the significant contamination caused by thick clouds. As a result, these methods face substantial challenges in maintaining semantic fidelity and capturing fine-grained structural details, often leading to geographically unfaithful features. In this paper, we propose a novel cloud removal approach, termed SRP-CR to tackle these challenges. Our proposed method effectively exploits semantic and representational priors from cloud-contaminated images, guiding the diffusion model to generate textures that are semantically consistent and rich in detail. To construct semantic priors, we utilize the advanced understanding and reasoning capabilities of Multi-Modal Large Language Models (MLLMs), which provide comprehensive global textual descriptions of RS images. These semantic descriptions are further enriched by integrating them with local visual feature representations derived from degraded RS imagery. For representational priors, we introduce the Dynamic Contextual Prior Fusion Module (DCPFM), which effectively captures and merges informative contextual data from multi-temporal cloudy images, thereby preserving low-level fidelity. Additionally, a semantic integration mechanism is devised to cohesively inject these acquired semantic priors into the diffusion model. Extensive experiments on three public datasets demonstrate the superior performance of the proposed method in comparison to existing state-of-the-art baselines.