A Method for Image Inpainting Considering Local Appearance Consistency

This paper proposes an image inpainting method which can preserve the local appearance consistency of reconstructed images. The proposed method is based on PICNet, an existing image inpainting method. PICNet is composed of two paths based on a VAE-GAN architecture: reconstructive and generative pipelines, and can outputs multiple reconstruction results with various appearances from an input image. The discriminators in both the pipelines, however, have a problem that they evaluate not the local but the global appearances of input images. The proposed method replaces the discriminator in the generative pipeline with SN-PatchGAN, a discriminator evaluating on local image features. Also, the proposed method replaces the convolution layers in the SN-PatchGAN with partial convolution layers applied only to masked regions. We expect to achieve image inpainting with local appearance consistency by these two improvements, and actually confirmed the effectiveness of the proposed method against the existing method through quantitative and qualitative evaluations.