KGViT: Knowledge Graph Integrated Vision Transformer for EEG-Based Emotion Recognition

Emotion recognition using electroencephalography (EEG) has attracted considerable attention and achieved significant advancements in the brain-computer interface field. Although many deep learning methods have shown superior performance over traditional approaches in emotion recognition tasks, existing studies neglect auxiliary information, such as participants’ details and descriptions of stimuli, focusing solely on EEG data. This oversight may lead to poor generalization across different participants. To address this issue, we propose a knowledge graph integrated with a vision transformer (KGViT). The KGViT model utilizes the vision transformer to capture effective features from the 3D temporal-spectral-spatial representations of EEG data. Innovatively, we design a knowledge graph feature extraction module using a graph convolutional network. This module processes node and relation matrices constructed from the EEG dataset, enabling the model to leverage prior knowledge for improved performance. The hybrid feature fusion module is introduced to calculate dynamic weights for knowledge graph features based on EEG features and integrate them into a unified representation. Extensive experiments were conducted on the DEAP and DREAMER datasets to assess the effectiveness of KGViT, with outcomes surpassing the state-of-the-art baseline. Accuracy rates of $\mathbf{9 7. 7 8 \%}, \mathbf{9 8. 0 6 \%}, \mathbf{9 8. 8 9 \%}$, and $\mathbf{9 7. 4 7 \%}$ were achieved on DEAP-Valence, DEAP-Arousal, DREAMER-Valence, and DREAMER-Arousal, respectively.