Audio-driven 3D Facial Animation with Multimodal Context Enhancement

Audio-driven 3D facial animation has emerged as a prominent research topic in computer graphics. Existing autoregressive methods are often slow and suffer from error accumulation, whereas non-autoregressive methods often struggle to establish global contextual dependencies. Moreover, current approaches remain insufficient in the cross-modal semantic alignment between audio and facial animation. To address these challenges, we propose a non-autoregressive audio-driven 3D facial animation method with multimodal context enhancement. Our method fully exploits multi-scale facial representations as well as the global contextual information within facial and audio feature sequences, enabling deep cross-modal fusion and precise semantic alignment. Specifically, we develop a global–local encoder built with spiral convolution, which facilitates complementary learning of global basic features and local fine-grained details, thereby improving the quality of facial feature extraction. We further design a multimodal context-enhanced cross-modal modulation module. This module adopts an Adaptive Instance Normalization (AdaIN)-based cross-modal fusion strategy to modulate facial features with audio sequence context in a single step, achieving fast generation without error accumulation. In addition, the module constructs a feature region cross-attention mechanism to capture contextual dependencies within facial features, and leverages globally-aware adaptive channel weighting to model temporal contextual relations of multimodal fused features. These designs enable effective exploitation of global context and substantially improve cross-modal semantic alignment accuracy. Extensive experiments demonstrate that our method outperforms mainstream baselines in both animation quality and generation efficiency.