EmotionFace: Speech-Driven Emotional 3D Face Animation Based on Facial Decoupling

Recent advancements in speech-driven 3D facial animation models have significantly enhanced the quality and realism of facial animations. However, existing methods often fail to adequately capture emotional expressions in videos or to effectively decouple facial features, limiting the ability to independently control emotional expressions. To address these limitations, we propose a novel model based on a large-scale visual models that integrates audio and textual style label and emotion source video prompts to independently control lip movements and emotional expressions. Specifically, we introduce the Video-Guided Emotional Motion Module, which leverages a large visual model to extract frame-level emotional features and maps them into an independent latent space. Furthermore, we present the Audio-Guided Lip Motion Module, which dynamically de-constructs lip movements and encodes them into a separate latent space. To enhance lip synchronization, we incorporate Avhubert technology, improving both accuracy and comprehensibility. Comprehensive experimental results demonstrate that our method not only generates high-quality facial animations but also authentically reflects emotional features, achieving superior performance compared to state-of-the-art techniques.