MD-GAN: Multi-Scale Diversity GAN for Large Masks Inpainting

Image inpainting approaches have made considerable progress with the assistance of generative adversarial networks (GANs) recently. However, current inpainting methods are incompetent in handling the cases with large masks and they generally suffer from unreasonable structure. We find that the main reason is the lack of an effective receptive field in the inpainting network. To alleviate this issue, we propose a new two-stage inpainting model called MD-GAN, which is a multi-scale diverse GAN. We inject dense combinations of dilated convolutions in multiple scales of inpainting networks to obtain more effective receptive fields. In fact, the result of inpainting large masks is generally not uniquely deterministic. To this end, we newly propose the multi-scale probabilistic diverse module, which achieves diverse content generation by spatial-adaptive normalization. Meanwhile, the convolutional block attention module is introduced to improve the ability to extract complex features. Perceptual diversity loss is added to enhance diversity. Extensive experiments on benchmark datasets including CelebA-HQ, Places2 and Paris Street View demonstrate that our approach is able to effectively inpaint diverse and structurally reasonable images.