Diffusion probabilistic model for Tibetan painted sketch extraction

During the process of Tibetan painting, the same sketch can be used to create different types of artwork. However, for each type, the artist must meticulously redraw an identical sketch. Crafting a complex sketch demands considerable time and effort from the artist, making the extraction of these sketchs crucial. Given the intricate colors and line structures of Tibetan paintings, existing learning-based methods for extracting sketchs often fail to accurately and clearly capture these elements. With the successful application of Diffusion Probabilistic Models (DPM), we propose a method for extracting lsketchs of Tibetan paintings based on DPM-DiffusionSketch. By introducing DPM, DiffusionSketch simultaneously captures coarse-grained global context and fine-grained local context features in two stages. We propose a corresponding adaptive wavelet filter to adjust latent features of specific frequencies and integrate the features extracted from both stages through a Feature Fusion Module (FFM). Moreover, to minimize errors and distortions in the generated sketchs, the fused output features are processed by a Generative Adversarial Network (GAN) to predict the final effect of the Tibetan painting sketchs. Through all the above technical designs, DiffusionSketch can generate clear and concise sketch of Tibetan paintings with minimal resource consumption. In our HeHuang Tibetan Painting (HHTP) dataset, the sketchs extracted by DiffusionSketch are less noisy, have clearer lines, and closely resemble the sketchs created by real Tibetan painting artists compared to existing sketch extraction methods. The sketchs extracted by our method rank first, with an average ranking of 1.137 among the results extracted by 36 different methods. On the HHTP dataset, our method outperforms the second-ranked method by 57.12 % in subjective evaluation and by 18.30 % in objective evaluation criteria. Code: https://github.com/HeHuangAI/DiffusionSketch .