An open dataset of multidimensional signals based on different speech patterns in pragmatic Mandarin

Speech is essential for human communication, but millions of people lose the ability to speak due to conditions such as amyotrophic lateral sclerosis (ALS) or stroke. Assistive technologies like brain-computer interfaces (BCIs), can convert brain signals into speech. However, these technologies still face challenges in decoding accuracy. This issue is especially challenging for tonal languages like Mandarin Chinese. Furthermore, most existing speech datasets are based on Indo-European languages, which hinders our understanding of how tonal information is encoded in the brain. To address this, we introduce a comprehensive open dataset, which includes multimodal signals from 30 subjects using Mandarin Chinese across overt, silent, and imagined speech modes, covering electroencephalogram (EEG), surface electromyogram (sEMG), and speech recordings. This dataset lays a valuable groundwork for exploring the neural encoding of tonal languages, investigating tone-related brain dynamics, and improving assistive communication strategies. It supports cross-linguistic speech processing research and contributes to data-driven neural speech decoding technology innovations.