CMDM: A Control Motion Diffusion Model for 2D Digital Human Motion Video Generation

In recent years, 2D digital human motion generation (DHMG) is becoming increasingly crucial for many areas such as virtual live broadcasting and film production. Although a lot of effort has been invested in DHMG, the complexity and diversity of human movements make it difficult to generate motions for arbitrary poses, and it is challenging to avoid distortion in the generated hand regions. To address these challenges, in this paper, we propose Control Motion Diffusion Model (CMDM), a diffusion based method for generating 2D digital human motion videos with arbitrary given poses. In CMDM, a novel Pose Control Module (PCM) is customized to fully encode pose and character information, generating multi-level and multi-resolution embedding, thereby ensuring that the model can learn more precise human motion features. In addition, data augmentation and VAE fine-tuning training strategies are adopted to decouple character and pose features, improving the model's ability to generate unseen poses. Finally, comprehensive experiments have been completed to demonstrate the effectiveness of the proposed method, and the proposed CMDM has achieved significant improvement in both subjective and objective evaluations.