R²-ESRGAN: Improved ESRGAN for Blind Image Super-Resolution

Existing blind super-resolution (SR) methods have fundamental limitations in reconstructing detailed texture and structure information lost during the degradation process because only a single LR image is used as input. To overcome this problem, we introduce the strategy of reference-based SR methods that utilize a high-resolution reference as an additional input. To this end, adopting ESRGAN as a baseline model as in previous blind SR studies, we add an image feature fusion module to the ESRGAN generator. In addition, we introduce the PatchGAN discriminator, Wasserstein GAN (WGAN) and WGAN-GP loss functions to reconstruct fine-grained texture and structure information clearly and reliably. In this paper, we call the improved ESRGAN R²-ESRGAN. Through experiments on images synthesized by applying various degradation processes and images with real degradations, it was confirmed that R²-ESRGAN can generate SR images with superior quantitative and qualitative image quality than previous blind SR methods. In particular, it was demonstrated that the use of reference images plays a very important role in improving blind SR performance.