Impact of Temporal Precision on Speech Synthesis Accuracy From Electrocorticographic Brain Signals

Accurately time-aligned spectral targets are essential for training electrocorticographic (ECoG) brain-computer interfaces (BCIs) intended for real-time speech output. This alignment is particularly challenging with "silent speech," in which speech occurs without phonation, or in the extreme, without articulation. Crafting suitably precise targets for silent speech is complex and error-prone due to multiple sources of temporal imprecision. We investigated how these temporal inaccuracies impact deep neural network performance in synthesizing speech from a BCI clinical trial participant who retained some speech capability. By simulating silent speech conditions through distortions in known acoustic target timings, we observed significant performance degradations at both phonetic and syllabic timescales. Accuracy-based measures offered a more reliable assessment of intelligibility than traditional metrics like the short-time objective intelligibility index, which may overestimate performance in low-precision contexts. These results underscore the need for advanced alignment techniques with precise phonetic and syllabic guarantees for silent speech BCIs focused on providing immediate output.