Accuracy-Weighted Late Fusion of Visual, Acoustic, and Linguistic Features for Deception Detection

Automated deception detection remains difficult because facial appearance, speech patterns, and language content vary across speakers and recording conditions.This study presents a decision-level framework that combines visual, acoustic, and linguistic evidence.The visual branch detects faces in sampled video frames, extracts SE-ResNet50 representations, and uses a Random Forest classifier.The acoustic branch joins eGeMAPSv02 and EmoBase descriptors from openSMILE and applies a multilayer perceptron.The linguistic branch uses RoBERTa-based representations with parameter-efficient adaptation and a Random Forest classifier.The framework combines the three probability outputs through weights proportional to the individual model accuracies.Experiments use the Real-Life Trial dataset containing 121 recordings, including 60 truthful and 61 deceptive clips.The strongest standalone audio, visual, and text configurations obtain 81.03%, 76.7%, and 71.1% accuracy, respectively.The weighted fusion obtains 86.78% accuracy, 0.9008 AUC, and 0.8678 macro F1.The study also examines how fusion changes individual predictions.Results support the usefulness of complementary evidence, while the small dataset and unresolved evaluation details limit claims about generalization.

Authors

Institutions

Publication Details

Journal
IARJSET
Published
2026-11-20
DOI
https://doi.org/10.17148/iarjset.2026.131169
Primary Topic
Deception detection and forensic psychology
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Accuracy-Weighted Late Fusion of Visual, Acoustic, and Linguistic Features for Deception Detection

Dineshkumar Bhagwandas Vaghela, Maheta Hardikkumar Harishbhai
2 links
IARJSET
Deception detection and forensic psychology
article

Accuracy-Weighted Late Fusion of Visual, Acoustic, and Linguistic Features for Deception Detection

Dineshkumar Bhagwandas Vaghela, Maheta Hardikkumar Harishbhai
article en

Abstract

Automated deception detection remains difficult because facial appearance, speech patterns, and language content vary across speakers and recording conditions.This study presents a decision-level framework that combines visual, acoustic, and linguistic evidence.The visual branch detects faces in sampled video frames, extracts SE-ResNet50 representations, and uses a Random Forest classifier.The acoustic branch joins eGeMAPSv02 and EmoBase descriptors from openSMILE and applies a multilayer perceptron.The linguistic branch uses RoBERTa-based representations with parameter-efficient adaptation and a Random Forest classifier.The framework combines the three probability outputs through weights proportional to the individual model accuracies.Experiments use the Real-Life Trial dataset containing 121 recordings, including 60 truthful and 61 deceptive clips.The strongest standalone audio, visual, and text configurations obtain 81.03%, 76.7%, and 71.1% accuracy, respectively.The weighted fusion obtains 86.78% accuracy, 0.9008 AUC, and 0.8678 macro F1.The study also examines how fusion changes individual predictions.Results support the usefulness of complementary evidence, while the small dataset and unresolved evaluation details limit claims about generalization.

IARJSETVol. 13(11)
Gujarat Technological University (IN)
Deception detection and forensic psychology
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon