MSA-TFN: Multi-Scale Attention Two-step Fusion Network for EEG-fNIRS Motor Imagery Classification

Motor imagery (MI) is crucial in brain-computer interface (BCI) research and is widely applied in neurorehabilitation and prosthetic control. Electroencephalogram (EEG) signals have high temporal resolution but are susceptible to noise and have low spatial resolution. In contrast, functional near-infrared spectroscopy (fNIRS) signals provide good spatial resolution but suffer from poor temporal resolution. Therefore, effectively combining the advantages of these two signals becomes a key issue in improving the performance of motor imagery classification. To address this challenge, this paper proposes a novel multi-scale attention two-step fusion network (MSA-TFN) that includes dual-path feature extraction, feature fusion, temporal modeling of fused features, and final classification. The dual-path feature extraction module captures EEG and fNIRS features independently, leveraging the advantages of both modalities. The multi-scale attention mechanism adaptively adjusts the fusion weights, and the two-step fusion strategy further enhances feature integration. Additionally, the Transformer module is introduced to capture long-range feature dependencies. This model was tested and evaluated using the publicly available TU-Berlin-A dataset, achieving an average accuracy of 83.29%, outperforming other state-of-the-art studies on the same dataset, which demonstrates the effectiveness of the proposed method.