LSGDM: shadow generation for composite image with latent diffusion model

Composite images generated through simply merging objects with background images often demonstrate a pronounced lack of realism due to the lack of shadows, an important visual element. Therefore, shadow generation plays a critical role within the domain of image synthesis. Existing shadow generation methods primarily employ Generative Adversarial Networks (GANs) as the underlying framework. However, GANs often overlook critical guiding features in background images due to mode collapse, leading to poor shadow generation quality. To address this concern, we have designed a two-stage shadow generation method based on Latent Diffusion Model (LDM). The non-adversarial training process allows the diffusion model to effectively avoid mode collapse. Specifically, in the first stage, we introduce a non-local attention mechanism and design a focal loss function based on Gaussian activation to generate high-quality shadow masks. In the second stage, we treat the shadow shading task as a special style transfer task to extract shadow shading guide features from background images, and design a partitioned dynamic weight loss to shade the masked areas as shadows. We demonstrate the effectiveness of our model through a series of experiments on the DESOBA dataset.