Filter bank common spatial pattern and envelope-based features in multimodal EEG-fTCD brain-computer interfaces

Brain-computer interfaces (BCIs) provide alternative means of communication and control for individuals with severe motor or speech impairments. Multimodal BCIs have been introduced recently to enhance the performance of BCIs utilizing single modality. In this paper, we aim to advance the state of the art in multimodal BCIs combining Electroencephalography (EEG) and functional transcranial Doppler ultrasound (fTCD) by introducing advanced analysis approaches that enhance system performance. Our EEG-fTCD BCIs employ two distinct paradigms to infer user intent: motor imagery (MI) and flickering mental rotation (MR)/word generation (WG) paradigms. In the MI paradigm, we introduce the use of Filter Bank Common Spatial Pattern (FBCSP) for the first time in an EEG-fTCD BCI, while in the flickering MR/WG paradigm, we extend FBCSP application to non-motor imagery tasks. Additionally, we extract previously unexplored time-series features from the envelope of fTCD signals, leveraging richer information from cerebral blood flow dynamics. Furthermore, we employ a Bayesian fusion framework that allows EEG and fTCD to contribute unequally to decision-making. The multimodal EEG-fTCD system achieved high classification accuracies across tasks in both paradigms. In the MI paradigm, accuracies of 94.53%, 94.9%, and 96.29% were achieved for left arm MI vs. baseline, right arm MI vs. baseline, and right arm MI vs. left arm MI, respectively - outperforming EEG-only accuracy by 3.87%, 3.80%, and 5.81%, respectively. In the MR/WG paradigm, the system achieved 95.27%, 85.93%, and 96.97% for MR vs. baseline, WG vs. baseline, and MR vs. WG, respectively, showing accuracy improvements of 2.28%, 4.95%, and 1.56%, respectively compared to EEG-only results. Overall, the proposed analysis approach improved classification accuracy for 5 out of 6 binary classification problems within the MI and MR/WG paradigms, with gains ranging from 0.64% to 9% compared to our previous EEG-fTCD studies. Additionally, our results demonstrate that EEG-fTCD BCIs with the proposed analysis techniques outperform multimodal EEG-fNIRS BCIs in both accuracy and speed, improving classification performance by 2.7% to 24.7% and reducing trial durations by 2-38 seconds. These findings highlight the potential of the proposed approach to advance assistive technologies and improve patient quality of life.