MSMGE-CNN: a multi-scale multi-graph embedding convolutional neural network for motor related EEG decoding

Abstract Deep learning technique has been widely used for decoding motor related electroencephalography (EEG) signals, which has considerably driven the development of motor related brain–computer interfaces (BCIs). However, traditional convolutional neural networks (CNNs) cannot fully represent spatial topology information and dynamic temporal characteristics of multi-channel EEG signals, resulting in limited decoding accuracy. To address such challenges, a novel multi-scale multi-graph embedding CNN (MSMGE-CNN) is proposed in this study. The proposed MSMGE-CNN contains two crucial components: multi-scale time convolution and multi-graph embedding. Specifically, we design a multi-branch CNN architecture with mixed-scale time convolutions based on EEGNet to sufficiently extract robust time domain features. Afterward, we embed multi-graph information obtained based on physical distance proximity and functional connectivity of multi-channel EEG signals into the time-domain features to capture rich spatial topological dependencies via multi-graph convolution operation. We extensively evaluated the proposed method on three benchmark EEG datasets commonly used for motor imagery/execution (MI/ME) classification and obtained accuracies of 79.59% (BCICIV-2a Dataset), 69.77% (OpenBMI Dataset) and 96.34% (High Gamma Dataset), respectively. These results powerfully demonstrate that MSMGE-CNN outperforms several state-of-the-art algorithms. In addition, we further conducted a series of ablation experiments to validate the rationality of our network architecture. Overall, the proposed MSMGE-CNN method dramatically improves the accuracy and robustness of MI/ME-EEG decoding, which can effectively enhance the performance of motor related BCI system.