XFashion: Character Animation Generation via Facial-enhanced and Granularly Controlling

Recent research has achieved advancements in animated fashion video synthesis. However, existing methods generate videos only with the guidance of poses, thus resulting in the generated characters appearing stiff and the expressions becoming monotonous. In this paper, we propose the XFashion, a diffusion-based method to generate vivid fashion videos, where the user could control the generation with portrait, emotion and poses. To achieve this, we design a prompt guider module to receive emotion-related prompts and encode them to the diffusion backbone to realize the granularly controlling for expressions and head-body coordination. Additionally, we devise a face-enhancing loss to further map the face regions and the reference prompts. We evaluate on two character animation benchmarks, and the results demonstrate that our proposed XFashion method effectively achieves stronger character animation generation results than state-of-the-art baselines.