Automated mural restoration via semi supervised segmentation and prompt guided diffusion inpainting

This study aims to address two major challenges in the digital restoration of murals: defect segmentation and large-scale defect repair. We propose a novel solution, combining the SAM-Adapter segmentation method with semi-supervised training and a Text-to-Inpainting strategy. In the aspect of defect segmentation, by introducing thresholding and semi-supervised training, we effectively enhanced the precision and applicability in handling sparse datasets. For large-scale defect repair, this study innovatively applies BPO (Black-box Prompt Optimization) technology, combined with GPT4V (ision) and Stable Diffusion, to achieve precise reconstruction of murals’ complex textures and structures through text-guided methods. Through extensive experimental validation, our method has demonstrated significant effectiveness in defect segmentation and large-scale repair, providing new perspectives and tools for the digital preservation of murals, and is of great significance for the protection of cultural heritage.