ECoG-Based Movement Classification and Limbs 3D Translation Prediction : a Deep Learning Study

Brain-Computer Interfaces (BCIs) aim at bridging residual neural activity related to motor intents with motor commands, presenting a revolutionary tool for helping tetraplegic and paraplegic individuals to regain motor control. Electrocorticography (ECoG)-based BCIs has emerged as a good compromise between invasiveness of the recording device and quality the recorded signals, making them a promising modality for BCI applications. The WIMAGINE system developed by CEA Clinatec for ECoG recording have been implanted for several years in proof of concept clinical trials on individuals with chronic impairments after severe spinal cord injury. Nevertheless, adequate algorithms are crucial to decipher brain signals ideally in real-time on portable systems so that these solutions can be proposed to patients in their daily life. While NPLS-based ECoG decoders have been successfully trained online in closed-loop clinical trials, deep learning alternatives have primarily been evaluated in offline settings. In this study, we investigate the offline training of deep learning models on datasets that replicate the temporal dynamics of real-time data acquisition. Our approach accounts for key factors such as inter-subject variability and signal drift, ensuring a more realistic evaluation. We systematically evaluate the performance of multiple models for classifying arm, leg, and wrist movements and predicting 3D translations of the arms and wrists, using varying amounts of ECoG data recorded from tetraplegic and paraplegic patients during motor imagery tasks. Additionally, we introduce a novel deep learning model based on the transformer architecture, specifically designed to be adjustable to scenarios with low data amounts. If on the considered classification datasets the NPLS-based ECoG decoder achieved better performances, on the considered regression datasets with the lowest data amounts, our model achieved performance comparable to, or exceeding, that of an NPLS decoder, while using less than half the parameters.