Diffusion Generation of Lace Texture Images with Self-Attention Mechanism Introduced in Higher Resolution Layers

Artificial design of lace images requires a lot of time and effort, and creativity is limited. Therefore, using generative models to assist in design becomes a good choice. However, the current lace generative models based on GAN (e.g. Stylegan2, etc.) have shortcomings in diversity and other aspects. Although current denoising diffusion models are able to generate more diverse images, the generated images suffer from texture loss and blurring. This article analyzes IDDPM (Improved Denoising Diffusion Probabilistic Models) and finds that its self-attention mechanism only focuses on the high-frequency components in the low-resolution layers (8x8, 16x16) of lace images. However, these high-frequency information are not sufficient to support good up-sampling effects, resulting in poor generation effect. In this paper, the self-attention mechanism is applied to the higher resolution layers (32x32, 64x64) to further enhance the high-frequency components in the high-resolution feature maps in the skip connections. In addition, to improve the up-sampling feature fusion effect of the model backbone network, this paper adopts quadratic linear interpolation as the up-sampling method. The analysis and experimental results show that the improved model generates lace images with rich details, maintaining symmetry and periodicity; in terms of diversity, the enhancement is obvious compared with the GAN method.