All-in-one mural restoration with prompt-guided residual diffusion

Ancient murals, as invaluable cultural heritage, are prone to degradation from natural erosion and human activities. Traditional manual restoration methods have inherent limitations, which in turn make virtual restoration a promising and innovative alternative. This paper thus proposes a diffusion-based virtual mural restoration method. To enable unified restoration of diverse degradation types, we first introduce a prompt-guided block. This block leverages the strong text feature extraction capability of pre-trained large language models to guide the extraction of mural image features. Secondly, we account for the semi-transparent nature of degradation patches. Damaged areas are not completely opaque, so we design a novel residual diffusion model. This model employs a prompt-guided UNet to predict semi-transparent residuals and time-dependent Gaussian noise. Our all-in-one model achieves the restoration of damaged murals across multiple dynasties, regions, and degradation types. Comprehensive experiments and ablation studies validate the method’s effectiveness, demonstrating that it achieves state-of-the-art performance and brings significant advancements to the field of ancient mural virtual restoration.