Comparison of Deep Learning-Based SAR-to-Optical Image Translation Models for High Spatial Resolution Optical Image Restoration

Despite the increased availability of high spatial resolution satellite images with high temporal resolution, including micro-satellite constellations, the restoration of missing regions due to clouds and cloud shadows in optical imagery remains crucial for constructing optical image time series.The translation of synthetic aperture radar (SAR) imagery into optical imagery, known as SAR-to-optical image translation, has been effectively applied for optical image restoration.However, few studies have applied SAR-to-optical image translation to restore missing regions in high spatial resolution optical imagery.This study evaluates the performance of SAR-to-optical image translation models using generative adversarial networks (GAN) for high spatial resolution optical image restoration.Three representative GAN-based models, including Pix2Pix, CycleGAN, and multi-temporal conditional GAN (MTcGAN), were selected in this study.MTcGAN, which utilizes additional multi-temporal SAR and optical image pairs, was particularly selected to investigate the effects of input images.SAR-to-optical image translation experiments were conducted using COSMO-SkyMed single-polarization images and multi-spectral PlanetScope images from the Gimje Plain area, with performance evaluation of predictions across various multi-temporal image pairs.The results showed that the spectral angle mapper values, which represent the multi-spectral band similarity, for Pix2Pix, CycleGAN, and MTcGAN were 9.1, 13.4, and 6.9respectively, indicating that MTcGAN generated predictions most spectrally similar to actual optical images.Furthermore, MTcGAN effectively preserved detailed structural information in both quantitative and qualitative evaluations.These findings suggest that incorporating additional input features in deep learning-based SAR-to-optical image translation can improve prediction accuracy.