EEG Lightformer-KD: a Lightweight Framework for Motor Imagery EEG Decoding via Knowledge Distillation

This study proposes a lightweight convolutional-Transformer hybrid architecture named EEG Lightformer, coupled with a knowledge distillation framework based on the EEGPT large model, to address deployment challenges of motor imagery decoding models in resource-constrained brain-computer interface systems. To address the contradiction between insufficient representation capability of conventional shallow convolutional networks and excessive parameters in large models, we innovatively design a two-stage knowledge transfer strategy: First, constructing a lightweight backbone network using depthwise separable convolutions and multi-scale classification mechanisms to compress parameter scale while preserving discriminative features; second, proposing a joint distillation loss function combining KL divergence distribution alignment with cross-entropy supervision to effectively transfer EEGPT’s spatiotemporal perception capabilities. Cross-session experiments on three public datasets (BCI Competition IV 2a/2b and OpenBMI) demonstrate that the distilled EEG Lightformer achieves 2.47-5.52 percentage point accuracy improvements compared to baseline models like EEGNet and FBCnet, while maintaining only 127K-205K parameters. The architecture attains 78.67% average accuracy in four-class tasks, reducing parameters by 84% compared to classical models while maintaining comparable performance. This framework provides a practical solution for wearable BCI systems that balances computational efficiency and decoding accuracy.