<b>Detecting Deception with Emotion: The Role of Emotional AI in Combating Social Engineering Attacks</b>

<b>Detecting Deception with Emotion: The Role of Emotional AI in Combating Social Engineering Attacks – Detailed Description</b>This study presents a novel, interdisciplinary framework integrating Emotional Artificial Intelligence (Emotional AI) into cybersecurity defenses to detect deceptive behavior during social engineering attacks. Social engineering remains one of the most elusive and dangerous forms of cyber intrusion, exploiting human psychology rather than technical flaws. While conventional cybersecurity systems are adept at defending against malware and network intrusions, they often fail to detect manipulative human tactics such as phishing, pretexting, and impersonation. This paper addresses that gap by exploring how affective signals can reveal deception and how these signals can be operationalized in AI-powered security systems.The proposed solution, titled Emotion-Aware Deception Detection (EADD), utilizes a multimodal emotion recognition architecture that analyzes facial expressions, vocal tones, and linguistic sentiment to detect emotional anomalies that correlate with deceptive intent. Drawing upon affective computing, natural language processing (NLP), and deep learning, the system captures subtle behavioral cues that humans might overlook—such as microexpressions, hesitation in voice, or inconsistencies between tone and language.The study employs the MDPE dataset (Multimodal Deceptive Personality Emotion dataset), which contains video, audio, and textual samples labeled as truthful or deceptive. EADD processes these through three deep learning pipelines:A ResNet50 model for facial emotion recognition,A wav2vec-based model for vocal stress and tone analysis,A RoBERTa NLP model for sentiment and linguistic deception markers.The outputs are fused using an early-fusion technique, enabling the system to learn complex interdependencies between modalities. The model is trained and validated using stratified k-fold cross-validation and evaluated using performance metrics including accuracy, precision, recall, F1-score, and ROC-AUC.Results show that the EADD system outperforms unimodal baselines, achieving an accuracy of 80.1%, F1-score of 0.80, and ROC-AUC of 0.85, demonstrating its viability for real-time deception detection. The paper also includes ablation studies to assess the contribution of each modality and comparative analysis with existing deception detection approaches.The discussion addresses ethical concerns, such as privacy, bias in emotion recognition, and the implications of deploying such systems in surveillance or border control. The paper recommends transparent model design, bias auditing, and human oversight when deploying Emotional AI in high-stakes environments.In conclusion, this research contributes to the growing field of affective cybersecurity, proposing a robust framework that bridges technical innovation with psychological insight. By transforming raw emotional signals into actionable security intelligence, the paper opens new possibilities for defending against socially engineered attacks in enterprise, governmental, and personal security settings.