Adaptive Multi-Granularity Information Exploration for EEG-Based Speech Recognition

Speech-related brain-computer interfaces (BCIs) have emerged as a promising paradigm for intuitive communication between human and external devices especially for people with language disorders. However, current Electroencephalogram (EEG)-based speech recognition performance remains inadequate for practical applications, primarily due to two challenges. One is the involvement of multiple brain networks in speech production, which renders single-domain features insufficient for comprehensive representation; the other is the low signal-to-noise ratio inherent in EEG data, coupled with variable data quality, which complicates decoding efforts. To address both issues, we propose an adaptive multi-granularity information exploration (AMIE) model for enhancing EEG-based speech decoding performance, which leverages complementary information across multiple feature domains and incorporates a tripartite dynamic weighting mechanism, taking all the domain, sample and feature importance into consideration, to improve model robustness and emphasize discriminative features. Experimental results on two public EEG data sets demonstrate the competitive performance in speech recognition as well as the effectiveness of the multi-granularity importance exploration.