STC-CapsNet: Detecting Audio Deepfakes with Spatio-Temporal Convolutions and Capsule Networks

Capsule networks are a powerful architecture designed to capture hierarchical relationships in data, making them effective for complex classification tasks. In audio deepfake detection, these networks effectively distinguish between real and synthetic audio by capturing subtle time and frequency patterns. Their ability to model intricate dependencies across both domains makes capsule networks especially suited for this challenging task. This study introduces the novel Spatio-Temporal Convolutional Capsule Network (STC-CapsNet), which utilizes mel-spectrograms and grayscale spectrograms for feature extraction. After preprocessing steps like noise reduction and segmentation, temporal and spectral convolutions are applied, followed by capsule layers with dynamic routing to enhance feature representation. The model is evaluated on the FoR dataset, achieving an F1-Score of 98.4% and a low EER of 2.8% using mel-spectrograms, outperforming grayscale spectrograms in both accuracy and error rate.