Edge Priors Image Inpaintig With <scp>StyleGAN2</scp>

ABSTRACT Image inpainting represents a fundamental task in computer vision, focusing primarily on the generation of missing content within an image to restore its integrity and aesthetics. Existing GAN‐based approaches often produce content with ambiguity and require a high training difficulties. Moreover, they tend to focus narrowly on damaged regions, leading to edge distortions that hinder generalisation. To address these challenges, we propose an algorithm that consist of two distinct networks. The first network, called Edge‐e4e, is designed for initial image restoration and integrates a pre‐trained StyleGAN2 as the generator to mitigate edge distortions. This network employs an encoder‐StyleGAN2 architecture, where only the encoder part is trained, thereby reducing training costs compared to traditional GAN methods. To resolve ambiguities in the restored content, we incorporate edge information into the damaged regions, guiding the network to generate content that is consistent with the original image. The second network, called Appending network, includes two style‐based encoders and a generator to improve the similarity between the images restored by Edge‐e4e and the original images. Specifically, we subtract the restored images from the input images in the channel dimension to obtain distortion maps, which serve as a prior to refine the restored images from Edge‐e4e. To further enhance the quality of refined images, we propose incorporating plugin and modulate plugin modules for style extraction and fusion. These modules utilise information from the input images and seamlessly integrate it into the style‐based generator. Experimental results demonstrate that our algorithm achieves high‐fidelity restoration and excellent generalisation, with optimal FID and Lpips metrics of 0.0631 and 0.875, respectively. The code is publicly available at: https://github.com/MengZhen‐Chi/Edge‐Pries‐Image‐Inpainting‐with‐StyleGAN2 .