EEG-Based Neurosteered Speaker Extraction in Cocktail Party Environment Without Stimulus Reconstruction

Previous studies on neurosteered hearing aids employed a neural decoder to reconstruct the speech stimulus from EEG signals to establish the attended sound source. However, this approach presents several limitations, such as the need for clean speech stimuli—which are often unavailable in real-world scenarios—and long processing windows. To address these challenges, we propose a novel EEG-based neurosteered speaker extraction (ENSE) mechanism that performs a joint action of speech separation and direct attention classification without the need for explicit speech stimulus reconstruction. Specifically, a typical speech separation model is first pretrained on a large speech corpus. We then train a speech-EEG match detector to perform direct attention classification by detecting which of the separated speech stimuli, or which of the speakers, induces the observed EEG signals. Experimental results show that ENSE effectively identifies and extracts the attended speech while suppressing unattended ones in a mixture. With time-domain speech separation and direct attention classification, ENSE offers a low-latency solution that marks an important step towards practical neurosteered hearing prostheses.