VAD-Lip: Visual and Audio Deepfake Detection via Lip Features

With the widespread adoption of internet technology and smart devices, video has become a key medium for information dissemination, permeating nearly every aspect of social life. However, the misuse of deepfake technology has raised several social issues, such as the creation of fake videos for manipulating public opinion, financial fraud, and social engineering attacks, severely disrupting social order. Existing studies primarily focus on pure visual feature analysis and multimodal detection methods, but detection of audio-video desynchronization forgery remains relatively underexplored. This study proposes a new large-scale audiovisual forgery dataset, the Audio-Video Lip-sync Forged Dataset (AVLiFD), and introduces a novel multimodal deepfake detection method, Visual and Audio Deepfake Detection via Lip Features (VAD-Lip). The method combines visual and audio features specifically for detecting deepfake videos forged through audio, video, and audiovisual joint manipulation leading to audio-visual desynchronization. Experimental results demonstrate that the proposed method achieves high detection accuracy for audio-video joint forgery, providing new data support and technical solutions for research in this area. On the AVLiFD dataset, our model achieved an accuracy of 92.50% and an AUC of 0.9518, outperforming other models. On the Celeb-DF, FF++, and DFDC datasets, it also achieved high accuracy rates of 94.67%, 93.67%, and 92.80%, respectively.