Accuracy-Weighted Late Fusion of Visual, Acoustic, and Linguistic Features for Deception Detection
Automated deception detection remains difficult because facial appearance, speech patterns, and language content vary across speakers and recording conditions.This study presents a decision-level framework that combines visual, acoustic, and linguistic evidence.The visual branch detects faces in sampled video frames, extracts SE-ResNet50 representations, and uses a Random Forest classifier.The acoustic branch joins eGeMAPSv02 and EmoBase descriptors from openSMILE and applies a multilayer perceptron.The linguistic branch uses RoBERTa-based representations with parameter-efficient adaptation and a Random Forest classifier.The framework combines the three probability outputs through weights proportional to the individual model accuracies.Experiments use the Real-Life Trial dataset containing 121 recordings, including 60 truthful and 61 deceptive clips.The strongest standalone audio, visual, and text configurations obtain 81.03%, 76.7%, and 71.1% accuracy, respectively.The weighted fusion obtains 86.78% accuracy, 0.9008 AUC, and 0.8678 macro F1.The study also examines how fusion changes individual predictions.Results support the usefulness of complementary evidence, while the small dataset and unresolved evaluation details limit claims about generalization.
Authors
- Dineshkumar Bhagwandas Vaghela (ORCID: https://orcid.org/0000-0002-4410-9513)
- Maheta Hardikkumar Harishbhai
Institutions
- Gujarat Technological University (IN)
Publication Details
- Journal
- IARJSET
- Published
- 2026-11-20
- DOI
- https://doi.org/10.17148/iarjset.2026.131169
- Primary Topic
- Deception detection and forensic psychology
- Type
- article
- Field-Weighted Citation Impact
- 0.00