GuidePaint: lossless image-guided diffusion model for ancient mural image restoration

Abstract Ancient murals, vital cultural heritage, suffer from damage due to natural erosion and human activities. Traditional restoration methods, relying on manual repair, have limitations, making virtual restoration an innovative solution. This paper proposes a virtual restoration method based on diffusion model. Using a lossless image-guided algorithm, we adapt diffusion model designed for image synthesis to restoration. Instead of feeding damaged images into the network, we use them to adjust the network’s outputs directly, achieving unsupervised training. We also use random seeds to generate diverse outputs from a single image. Proposed similarity function ensures alignment of undamaged areas with the guiding image, and an interrupt sampling strategy removes subtle, dense degradations. Experiments on simulated and real damaged murals show our method yields results comparable to or better than other advanced methods for simple cases. For complex and severely damaged murals, it excels, outperforming others in both objective and subjective evaluations.