Enhancing Cloud-Removed Regions in Multispectral Optical Images Using SAR Edge Features

Since the rapid development of deep learning, the task of cloud removal in multispectral optical (MSO) images has been revisited and reconsidered from new perspectives. However, due to insufficient attention from researchers, the study of Synthetic Aperture Radar (SAR)-based cloud removal methods based on deep learning remains in its early stages. Crude image-level fusion strategies are often adapted for SAR-based MSO image cloud removal. This study proposes a novel Transformer-based cloud removal network (TCRNet) utilizing SAR edge features to enhance the recovered region in MSO images. TCRNet models each spectral channel as a token, enabling efficient self-attention computations across the spectral dimension. This strategy empowers the proposed model to exploit the global spectral features captured by MSO sensors. A SAR edge feature extraction module is designed to extract multi-scale edge prior maps. The resulting features are seamlessly integrated into the U-shape MSO backbone network using spectral-wise multi-head self-attention, elevating attention to the critical edge regions during feature fusion. During the training process, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L_mae</i> loss constrains the pixel-level approximation, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L_per</i> spatial perceptual loss enhances the similarity of texture features between the predicted cloud-free and ground truth MSO images, regardless of cloud coverage. Experiments conducted on the SEN12MS-CR Dataset show that TCRNet outperforms state-of-the-art methods. The source code will be released at https://github.com/MinghuaWang123/TCRNet.