A Hierarchical Temporal Convolution Network for ERP Detection Based on an Auditory-Tactile Brain-Computer Interface System

By delivering sensory stimuli in an oddball paradigm, the brain-computer interface (BCI) is able to capture subject’s electroencephalogram (EEG) signals and analyze it. While most P300-based BCIs rely on visual stimuli, prolonged visual attention may lead to fatigue and limit the applicability for patients with visual impairment. This study proposes a BCI system integrating spatial auditory and electro-tactile stimuli. The system employs a tailored Hierarchical Temporal Convolutional Network (HTCN) to improve the performance of event-related potential (ERP) detection. Six healthy controls were involved in BCI experiment for comparing different modalities, followed by exploratory BCI assessments with two visually impaired patients. Results show that healthy controls performed better in auditory-tactile (AT) modality than in auditory (A) and tactile (T) modalities, with an average online accuracy of 77.17% ± 4.84%. Furthermore, the designed HTCN achieved higher average offline accuracy compared to other approaches. These findings suggest that our proposed BCI system effectively enhances EEG effects in subjects, and it has potential in visual-independent clinical BCI assessment.