Semantic and Sketch-Guided Diffusion Model for Fine-Grained Restoration of Damaged Ancient Paintings

Ancient paintings, as invaluable cultural heritage, often suffer from damages like creases, mold, and missing regions. Current restoration methods, while effective for natural images, struggle with the fine-grained control required for ancient paintings’ artistic styles and brushstroke patterns. We propose the Semantic and Sketch-Guided Restoration (SSGR) framework, which uses pixel-level semantic maps to restore missing and mold-affected areas and depth-aware sketch maps to ensure texture continuity in creased regions. The sketch maps are automatically extracted using advanced methods that preserve original brushstroke styles while conveying geometry and semantics. SSGR employs a semantic segmentation network to categorize painting regions and depth-sensitive sketch extraction to guide a diffusion model. To enhance style controllability, we cluster diverse attributes of landscape paintings and incorporate a Semantic-Sketch-Attribute-Normalization (SSAN) block that explores consistent patterns across styles through spatial semantic and attribute-adaptive normalization modules. Evaluated on the CLP-2K dataset, SSGR achieves an mIoU of 53.30%, SSIM of 0.42, and PSNR of 13.11, outperforming state-of-the-art methods. This approach not only preserves historical aesthetics but also advances digital heritage preservation with a tailored, controllable technique for ancient paintings.