GRS-Wav2lip: General Speech-Driven Lip Synthesis in the Wild

In recent years, advancements in virtual digital human technology have accelerated considerably; however, many existing lip synchronization models lack adequate focus on lip details, often leading to asynchrony between speech and lip movements. This paper introduces a novel method, GRS- Wav2Lip, designed to enhance the naturalness and synchronization of generated talking face videos. We have restructured the convolutional layers of the identity encoder within the generator, replacing standard convolutional layers with gated convolutions to improve the model's ability to capture lip-related details and motion-specific features effectively. Additionally, we introduce the SAC module to dynamically adjust the receptive field, enabling the effective capture of both global and local features. To improve resolution and image detail, we integrate the RRDB module, which significantly enhances image quality even with low-resolution inputs. Finally, we employ the LPIPS loss function to accurately measure the perceptual quality of generated images. Experimental results show that our approach not only strengthens the model's generative capacity but also leads to substantial improvements in lip synchronization and image detail quality.