Pose Transfer with Multi-Scale Features Combined with Latent Diffusion Model and ControlNet

In recent years, generative AI has become popular in areas like natural language processing, as well as image and audio processing, significantly expanding AI’s creative capabilities. Particularly in the realm of image generation, diffusion models have achieved remarkable success across various applications, such as image synthesis and transformation. However, traditional diffusion models operate at the pixel level when learning image features, which inevitably demands significant computational resources. To address this issue, this paper proposes a pose transfer model that integrates the latent diffusion model, ControlNet, and a multi-scale feature extraction module. Moreover, the proposed method incorporates a semantic extraction filter into the attention neural network layer. This approach enables the model to train images in the latent space, subsequently focusing on critical image features and the relationships between poses. As a result, the architecture can be efficiently trained using an RTX 4090 GPU instead of multiple A100 GPUs. This study advances generative AI by optimizing diffusion models for enhanced efficiency and scalability. Our integrated approach reduces computational demands and accelerates training, making advanced image generation more accessible to organizations with limited resources and paving the way for future innovations in AI efficiency.