Spatiotemporal Fusion for Deepfakes Detection in Face Videos

Deep forgery, a deep learning-based image tampering technique, poses significant challenges to existing detection methods, which primarily focus on identifying spatial artifacts within individual video frames. However, as deep forgery techniques advance, capturing these spatial inconsistencies has become increasingly difficult. To address this, we shift our focus to detecting spatiotemporal features that exist across consecutive video frames. When extracting temporal features, traditional Transformer models struggle to fully capture the temporal dependencies inherent in time series data. We use TimeSformer to extract spatiotemporal features from video frames, which are then fused and used for detection. To further enhance feature extraction, we integrate the SE (Squeeze-and-Excitation) module into the Middle Flow of the Xception network. This module uses global information to highlight important features and ignore irrelevant ones. To test our method's effectiveness, we conducted comparisons using the FaceForensics++ dataset and performed cross-dataset evaluations with the Celeb-DF dataset. Experimental results show that our detection method achieves 96.12% accuracy on the FaceForensics++ dataset, surpassing other methods. Additionally, the cross-dataset test yields an accuracy of 73.73%, showcasing a degree of generalization across different datasets.