Improving cognitive stress classification via multimodal EEG and ECG fusion: gender differences in physiological response

In this paper, we present a multimodal machine learning system integrating electroencephalogram (EEG) and electrocardiogram (ECG) features to discriminate cognitive stress levels and analyze gender differences. In particular, we assess how well feature fusion enhances the classification accuracy and we also compare the physiological signs of cognitive stress in male and female subjects. A publicly available dataset, namely, ECG and EEG features for mental workload and multilevel stress classification of different sexes, was used. Three essential physiological characteristics (Theta/Alpha Ratio (TAR), Heart Rate (HR), and LF/HF Ratio) were identified in EEG signals as well as in ECG signals. The discriminative power of the features was confirmed by statistical testing, such as Shapiro-Wilk and Kruskal-Wallis tests. Dimensional reduction was performed using Principal Component Analysis (PCA). Feature fusion was performed, where EEG and ECG signals were introduced and aligned to form a complete input for the classification. Six machine learning (ML) classification models, namely Decision Tree (DT), K-Nearest Neighbors (KNN), Linear Discriminant (LD), Naive Bayes (NB), Random Forest (RF), and Support Vector Machine (SVM), were tested on binary and multiclass cognitive stress tasks. In terms of overall performance, the SVM achieved the best results, attaining 94.7% accuracy in the combined-gender classification of cognitive stress. For sex-specific classifications, LD recorded the highest score (90.9%). The multimodal SVM demonstrated superiority over the unimodal EEG/ECG models, reaching a peak average accuracy of 92.6%. Owing to the gender-based analysis, females had a better score in classification, implying a difference in sex-related physiological patterns during cognitive stress. The results show that multimodal fusion of physiological features can improve cognitive stress classification. The suggestion of gender-based analysis highlights the possibility of cognitive monitoring systems being at a personal level. This has potential applications in education, ergonomics, and human-computer interaction.