Impact of sex differences on subject-independent EEG-based emotion recognition models

Electroencephalography (EEG) can capture emotion regulation by recording electrical activity from the brain cortex. This electrical activity can be processed using deep learning models to predict emotional states. Two approaches can be employed to develop these deep learning models: subject-dependent and subject-independent. The subject-independent approach is more practical since it does not require recalibration for new users. However, it faces challenges due to the high variability of EEG signals among individuals. Recent studies suggest that incorporating subjects' sex can enhance the accuracy of subject-independent models due to differences in emotional processing between males and females. Although previous studies have demonstrated the effect of sex on emotion recognition, they have primarily focused on the predictive aspect, neglecting the interpretability of how emotion regulation differs between males and females. This work addresses this limitation by using attention network layers to identify brain areas more involved in predicting emotions. Additionally, an odds ratio analysis was conducted using logistic regression to evaluate the impact of sex on emotion prediction. Our findings reveal that cortical activation patterns elicited by emotional audio-visual stimuli differ between females and males, with females showing more neural activation in the left hemisphere and males showing more in the right hemisphere. Moreover, when the output probabilities of the deep learning models are further postprocessing with the subject's sex, the odds of correctly predicting emotions increase. These findings suggest that sex differences can lead to more robust subject-independent emotion recognition models.