CartoonTalk: A speech-driven animation model for cartoon face generation

Due to the substantial differences in appearance<br/>and movement between cartoon faces and real faces, directly<br/>applying existing speech-driven speaker generation techniques to<br/>cartoon face images often fails to accurately capture key points,<br/>resulting in distorted expressions and animation artifacts. To<br/>address these challenges, this paper proposes a speech-driven<br/>cartoon face animation generation model called CartoonTalk,<br/>which combines 2D cartoon key points and 3D motion coefficients.<br/>The model is divided into three modules: face annotation,<br/>motion extraction, and 3D facial rendering. The face annotation<br/>module first annotates a cartoon image and captures its 2D key<br/>points. Subsequently, the motion extraction module extracts 3D<br/>motion coefficients for head and facial expressions from speech<br/>data. Finally, these 2D and 3D features are fed into the 3D<br/>facial rendering module to generate the animated cartoon face.<br/>Quantitative and qualitative evaluations reveal that our model<br/>significantly outperforms existing approaches in both lip-sync<br/>accuracy and visual quality, establishing new state-of-the-art<br/>benchmarks on the cartoon datasets.