Deepfake Detection Using Multi-Modal Fusion Combined with Attention Mechanism

The proliferation of deepfake technology poses a significant challenge to the authenticity of digital content. This research explores the application of multimodal fusion techniques to enhance deepfake detection accuracy. By combining visual and audio features, the proposed method leverages the complementary nature of different data types to detect discrepancies introduced by deepfake manipulation. An attention mechanism is incorporated to focus on salient regions within each modality, further improving detection accuracy. Convolutional Neural Networks (CNNs) and Mel-Frequency Cepstral Coefficients (MFCCs) are employed for feature extraction, followed by feature fusion for deepfake detection. This approach demonstrates the effectiveness of multimodal fusion in combating the evolving threat of deepfake technology. By advancing deepfake detection techniques, this research contributes to safeguarding the integrity of digital content and preserving trust in media.

Authors

Institutions

Publication Details

Published
2024-10-15
DOI
https://doi.org/10.1109/icses63445.2024.10763221
Citations
3
Primary Topic
Anomaly Detection Techniques and Applications
Type
article
Field-Weighted Citation Impact
1.04
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Deepfake Detection Using Multi-Modal Fusion Combined with Attention Mechanism

C. P. Shirley, R. Venkatesan, M B Abisha, Elakkiya Elango et al.
3 citations
Anomaly Detection Techniques and Applications
1.04
article

Deepfake Detection Using Multi-Modal Fusion Combined with Attention Mechanism

C. P. Shirley, R. Venkatesan, M B Abisha, Elakkiya Elango, Berin Jeba Jingle, Yashvanth Ram R V
article en
3 citations

Abstract

The proliferation of deepfake technology poses a significant challenge to the authenticity of digital content. This research explores the application of multimodal fusion techniques to enhance deepfake detection accuracy. By combining visual and audio features, the proposed method leverages the complementary nature of different data types to detect discrepancies introduced by deepfake manipulation. An attention mechanism is incorporated to focus on salient regions within each modality, further improving detection accuracy. Convolutional Neural Networks (CNNs) and Mel-Frequency Cepstral Coefficients (MFCCs) are employed for feature extraction, followed by feature fusion for deepfake detection. This approach demonstrates the effectiveness of multimodal fusion in combating the evolving threat of deepfake technology. By advancing deepfake detection techniques, this research contributes to safeguarding the integrity of digital content and preserving trust in media.

Karunya University (IN)
Openalex Percentile: Top 18%
Anomaly Detection Techniques and Applications
1.04
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.