NeuroStrainSense: A Transformer-Generative AI Framework for Stress Detection Using Heterogeneous Multimodal Datasets

Stress is a pervasive global health concern that adversely contributes to morbidity and reduced productivity, yet it often remains unquantified due to its subjective and variant presentation. Although artificial intelligence offers an encouraging path toward automated monitoring of mental states, current state-of-the-art approaches are challenged by the reliance on single-source data, sparsity of labeled samples, and significant class imbalance. This paper proposes NeuroStrainSense, a novel deep multimodal stress detection model that integrates three complementary datasets—WESAD, SWELL-KW, and TILES—through a Transformer-based feature fusion architecture combined with a Variational Autoencoder for generative data augmentation. The Transformer architecture employs four encoder layers with eight multi-head attention heads and a hidden dimension of 512 to capture complex inter-modal dependencies across physiological, audio, and behavioral modalities. Our experiments demonstrate that NeuroStrainSense achieves a state-of-the-art performance with accuracies of 87.1%, 88.5%, and 89.8% on the respective datasets, with F1-scores exceeding 0.85 and AUCs greater than 0.89, representing improvements of 2.6–6.6 percentage points over existing baselines. We propose a robust evaluation framework that quantifies discrimination among stress types through clustering validity metrics, achieving a Silhouette Score of 0.75 and Intraclass Correlation Coefficient of 0.76. Comprehensive ablation experiments confirm the utility of each modality and the VAE augmentation module, with physiological features contributing most significantly (average performance decrease of 5.8% when removed), followed by audio (2.8%) and behavioral features (2.1%). Statistical validation confirms all findings at the p < 0.01 significance level. Beyond binary classification, the model identifies five clinically relevant stress profiles—Cognitive Overload, Burnout, Acute Stress, Psychosomatic, and Low-Grade Chronic—with an expert concordance of Cohen’s κ = 0.71 (p < 0.001), demonstrating the strong ecological validity for personalized well-being and occupational health applications. External validation on the MIT Reality Mining dataset confirms the generalizability with minimal performance degradation (accuracy: 0.785, F1-score: 0.752, AUC: 0.849). This work underlines the potential of integrated multimodal learning and demographically aware generative AI for continuous, precise, and fair stress monitoring across diverse populations and environmental contexts.