Vividhead: Speech-Driven 3D Facial Animation with Head Pose Using Decoupled Speech Feature

Recently, speech-driven 3D head animation has advanced, yet most approaches focus on either acoustic or semantic details of speech, neglecting their combined potential. Furthermore, head pose, crucial for natural interaction, is often ignored. We introduce VividHead, a framework that decouples speech features to enhance facial and head pose animations. For facial animation, a dual-path network employs VQ-VAE with contrastive learning for lip movements and an autoregressive model for global facial dynamics. For head pose, a crossdecoupling strategy links specific speech features to head movements, boosting diversity and naturalness. Our approach significantly improves lip-sync and head pose diversity, outperforming existing methods in both qualitative and quantitative analyses. We provide the codes and supplementary material with a GitHub repository link. https://github.com/Zrduan724/VividHead.