HSA-Former: Hierarchical Spatial Aggregation Transformer for EEG-Based Emotion Recognition

Affective brain–computer interfaces (aBCIs) have shown promising applications due to the significant advancements in utilizing electroencephalogram (EEG) signals for emotion recognition. By measuring neuronal activity across various brain regions, EEG provides rich spatial information that is essential for discriminative feature extraction and effective emotion recognition. However, existing deep learning models face challenges in effectively capturing and leveraging local and global spatial dependencies. To address this, we propose a hierarchical spatial aggregation Transformer (HSA-Former) for EEG-based emotion recognition, which explores spatial relationships from multiple levels, including electrodes, intrabrain regions, and interbrain regions. The HSA-Former is characterized by: 1) a multihierarchical spatial information learning architecture that sequentially extracts diverse spatial features from electrodes, through intrabrain regions, and across interbrain regions; 2) a parallel learning approach that captures internal spatial features of different brain regions from different channels; and 3) an effective aggregation method that mitigates the problem of information loss typically caused by direct pooling in Transformers. Extensive experiments on three public datasets (i.e., SEED, SEED-IV, and SEED-V) demonstrate that the proposed HSA-Former outperforms existing state-of-the-art methods. Interestingly, the weights learned by HSA-Former emphasize the frontal, temporal, and occipital lobes as critical brain regions, aligning with established mechanisms underlying emotion recognition.