Hybrid 3D-2D Deep Multi-Source Fusion Framework for Cloud Removal From SAR-Optical Data

Technological developments in satellite imagery sparked innovative advances in Earth observation efforts. The ability to identify land use and cover, monitor agriculture, detect changes, and handle disasters is made possible by remotely sensed imagery’s high spatial and temporal resolution. Aerosols and clouds, for example, often impede space observations, which is undesirable for computer-based systems that heavily depend on the data that is accessible. Although the issue of cloud removal has been researched for decades, quality reconstructions still need to be improved due to the unpredictability of cloud presence and kind. The cloud reconstruction problem was guided by combining optical data with auxiliary information from Synthetic Aperture Radar (SAR) systems. This research introduces a hybrid 3D-2D Deep Fusion Framework, which can remove clouds from Sentinel-2 data by incorporating invaluable morphological understandings from co-registered Sentinel-1 SAR. The proposed architecture eliminates cloud occlusion, regardless of thickness or density. The model is evaluated against SEN12MS-CR, a global real fusion-based cloud-removal dataset. The network performs better than state-of-the-art methods when measured against them.