FastTalk: Speech-Driven Lip Synchronization Video Generation for Chinese-Language Scenarios

Speech-driven lip synchronization is an important technique for talking-face video generation, with broad application potential in virtual humans, video dubbing, digital media, and human–computer interaction. However, existing methods still face challenges in achieving reliable lip synchronization while maintaining stable identity preservation, high visual fidelity, and efficient inference, especially in Chinese-language scenarios where related research remains relatively limited. To address these issues, we propose FastTalk, a speech-driven lip synchronization method for Chinese-language scenarios. The proposed framework performs latent-space restoration for efficient video synthesis, uses a fixed-mask strategy to suppress shortcut visual cues and strengthen audio-driven lip-shape prediction, and adopts a two-stage training scheme to reduce the gap between training and inference. This design improves generation stability while preserving efficiency. Experimental results show that FastTalk achieves competitive lip synchronization performance while improving visual quality and identity preservation. These results indicate that FastTalk provides an effective solution for Chinese speech-driven lip synchronization video generation.

Authors

Institutions

Publication Details

Journal
Applied Sciences
Published
2026-05-01
DOI
https://doi.org/10.3390/app16094438
Primary Topic
Speech and Audio Processing
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

FastTalk: Speech-Driven Lip Synchronization Video Generation for Chinese-Language Scenarios

Tao Fan, Xu Zhao, Guozhong Wang, Yizhang Liu
Applied Sciences
Speech and Audio Processing
article

FastTalk: Speech-Driven Lip Synchronization Video Generation for Chinese-Language Scenarios

Tao Fan, Xu Zhao, Guozhong Wang, Yizhang Liu
article en

Abstract

Speech-driven lip synchronization is an important technique for talking-face video generation, with broad application potential in virtual humans, video dubbing, digital media, and human–computer interaction. However, existing methods still face challenges in achieving reliable lip synchronization while maintaining stable identity preservation, high visual fidelity, and efficient inference, especially in Chinese-language scenarios where related research remains relatively limited. To address these issues, we propose FastTalk, a speech-driven lip synchronization method for Chinese-language scenarios. The proposed framework performs latent-space restoration for efficient video synthesis, uses a fixed-mask strategy to suppress shortcut visual cues and strengthen audio-driven lip-shape prediction, and adopts a two-stage training scheme to reduce the gap between training and inference. This design improves generation stability while preserving efficiency. Experimental results show that FastTalk achieves competitive lip synchronization performance while improving visual quality and identity preservation. These results indicate that FastTalk provides an effective solution for Chinese speech-driven lip synchronization video generation.

Applied SciencesVol. 16(9)
Shanghai University of Engineering Science (CN)
Quality Education
Openalex Percentile: Top 6%
Speech and Audio Processing
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.