STRFLNet: Spatio-Temporal Representation Fusion Learning Network for EEG-Based Emotion Recognition

Electroencephalography (EEG)-based emotion recognition is essential for medical assistance and human-computer interaction. Although deep learning-based emotion recognition methods have demonstrated high performance, several challenges remain: 1) How to effectively utilize the complex dynamic-static spatial patterns inherent in emotion-related EEG signals. 2) How to hierarchically learn the latent correlations among multi-domain features. To address these challenges, a spatio-temporal representation fusion learning network (STRFLNet) is proposed to improve both the accuracy and robustness of emotion recognition using EEG signals. Specifically, dynamic-static graph topologies are constructed to capture comprehensive brain functional connectivity states, and a continuous dynamic-static graph ordinary differential equations is introduce to reveal continuous spatial patterns within EEG signals. Additionally, a hierarchical transformer fusion module is developed to fully leverage the latent correlations among multi-domain features to obtain the fused spatio-temporal representation. The experimental results on the SEED, SEED-IV, and DREAMER public EEG emotion datasets, under both subject-independent and subject-dependent settings, demonstrate that STRFLNet outperforms state-of-the-art methods in emotion recognition tasks. We further validate the effectiveness of the proposed model through interpretability analysis, which reveals the associations between the activated brain regions and corresponding emotional states. Our work highlights the significance of continuous spatial pattern learning and spatio-temporal feature fusion in emotion recognition, providing new insights for EEG-based emotion modeling.