Real-Time EEG-Based BCI for Self-Paced Motor Imagery and Motor Execution Using Functional Neural Networks

This paper introduces a novel application of functional neural networks (FNNs) in the domain of electroencephalography-based (EEG-based) brain-computer interfaces (BCIs), targeting self-paced motor execution (ME) and motor imagery (MI). FNNs represent a neural network architecture tailored to smooth processes, and as such have been applied to EEG data classification recently. The paper proposes a comprehensive pipeline encompassing data acquisition, synchronization, pre-processing, training of FNNs, and real-time inference to enable the seamless integration of FNNs into real-world BCI applications. For the first time, FNNs are integrated into an end-to-end pipeline and serve for live inference outside a strict laboratory setting. In pursuit of a more accessible electroencephalography (EEG) artificial intelligence (AI) training scenario, the paper introduces a self-paced environment for auto-labeling EEG data. A custom-designed Pong game serves as the training task and enables subjects to engage in MI/ME tasks while receiving immediate visual feedback. To automate the labeling process of the recorded EEG data, the movements of both arms are captured with inertial measurement units (IMUs) and analyzed through gesture recognition. This novel training framework contributes to more natural and engaging data collection and reduces pre-processing for model training. To provide a comprehensive evaluation, the paper compares the performance of FNN and EEGNet in the self-paced MI/ME tasks. The comparative analysis addresses factors such as classification accuracy, real-time processing speed, and power consumption. Furthermore, the study explores various auto-labeling methods within the self-paced environment, analyzing their impact on the classification performances of both architectures. By evaluating these labeling methods, this work addresses the challenge of accurate and efficient EEG data labeling, crucial for training robust models for prediction of time critical events. The proposed pipeline and experimental design culminate in a full-scale evaluation of the FNN-based classification system to demonstrate its efficacy in real-time MI/ME tasks. The paper’s contributions not only establish FNNs as a potent tool for EEG classification but also provide valuable insights into enhancing the accessibility, usability, and performance of EEG-based AI systems in real-world applications.