Progressive artistic image and multispectral image inpainting via multi-scale diffusion and dense contrastive learning

Chinese paintings may contain important information that carry cultural and heritage values. However, due to various reasons such as natural disasters (earthquakes), some paintings are more or less damaged. Though existing inpainting methods for natural images have shown great success, they cannot be applied directly to artistic image inpainting. Therefore, the inpainting of artistic images is still a challenging task. As an important supplement to traditional cultural heritage digitization methods, the use of multispectral technology to digitize cultural relics enables the transformation of cultural relics from their physical form to a digital form, laying the foundation for the analysis and visualization of cultural heritage information. To this end, we introduce a progressive image inpainting framework based on multi-scale diffusion and dense contrastive learning. Specifically, we first repair the damaged image in the semantic space of different scales (micro, meso, and acro), which is implemented through the multi-scale diffusion model. Then, the inpainting results in different semantic spaces are used as guidance and fed to the corresponding decoder layer to achieve the final high-quality artistic image inpainting in the pixel space. To take advantage of the content self-similarity of artistic images, we employ the dense contrastive learning loss function to enhance the quality of inpainting further. We conduct experiments on dataset of ancient Chinese paintings and multispectral images built by ourselves, which show that our proposed framework achieves state-of-the-art performance.