Cross-domain multimodal learning for stress-level prediction: a hybrid deep learning framework integrating independent EEG and facial expression datasets

Mental stress detection requires robust classification systems capable of leveraging complementary physiological and behavioral indicators. This study presents a cross-domain multimodal framework that integrates independent benchmark datasets to predict stress levels through neurophysiological and facial expression analysis. The proposed architecture employs a dual-branch approach: Long Short-Term Memory (LSTM) networks process temporal EEG patterns from the DEAP dataset, while a hybrid Vision Transformer-Convolutional Neural Network (ViT-CNN) model extracts facial features from FER2013. Three fusion strategies are systematically evaluated to address cross-domain data integration challenges: early fusion via feature concatenation, late fusion through ensemble decision-making, and stacked fusion using meta-classifier architecture. The framework demonstrates exceptional generalizability across independent data sources, with stacked fusion achieving 91.64% accuracy, 0.93 precision, 0.94 recall, and 0.95 F1-score for three-level stress classification (low, moderate, high). Comparative analysis validates the superiority of cross-domain fusion over existing unimodal approaches, demonstrating significant performance improvements without synchronized data collection. This methodology addresses practical deployment scenarios where simultaneous multimodal recording is not possible all the time , offering substantial potential for real-world stress monitoring in healthcare, occupational environments, and intelligent human-computer interaction systems.