Lip Synchronisation for Mobile Based on Dynamic Sub-band Weighting

D digital human technology involves high computational requirements such as real-time rendering and lip synchronisation (LipSync), and the Discrete Cosine Transform (DCT) is the preferred choice for mobile deployment due to its low computational burden. However, the inaccuracy of DCT in consonantal lip synchronisation generation makes digital human expressions lack of details. To address this challenge, this paper proposes a lip synchronisation algorithm that combines DCT with dynamic sub-band weighting. Through the Blendshape mapping strategy with dual pathways for vowels and consonants, this method not only improves the accuracy of lip synchronisation, but also maintains the computational performance advantage over the traditional Fast Fourier Transform (FFT) method, which makes it more suitable for mobile 3D digital human application scenarios.