Co-speech holistic 3D motion generation with style from video

Speech-driven 3D motion generation has garnered increasing research attention. However, it faces significant challenges in achieving style controllability, primarily due to the scarcity of motion style annotations. To address this, we propose a novel diffusion-based framework for co-speech holistic motion generation that enables example-based style control from videos. Our approach integrates hierarchical speech encoding with rhythm-aware denoising to produce natural and synchronized gestures and expressions. For effective style guidance, we introduce a contrastive style encoder that captures discriminative style representations from reference clips without explicit labeling, enabling generalization to motion styles unseen during training. Furthermore, we design a neural mapper that aligns 2D and 3D gesture features in a shared embedding space, facilitating direct style extraction from in-the-wild videos and seamless transfer to 3D motion. Extensive experiments and user studies show that our proposed approach achieves state-of-the-art performance in both qualitative and quantitative evaluations, offering a flexible solution for controllable motion generation.