251 Towards a Real-Time Speech Neuroprosthesis to Restore Speech in a Person With ALS

INTRODUCTION: Communication is a top priority for people living with dysarthria. One approach to restoring communication is to use a brain-computer interface (BCI), a system that links the neural activity to external devices. By translating intracortical neural activity to text appearing on a screen, we have previously shown how a man with ALS and severe dysarthria could use a BCI to communicate at 98% accuracy from a dictionary of 125,000 words. METHODS: As part of the ongoing BrainGate2 clinical trial (NCT00912041), we surgically implanted four 64-channel microelectrode (Utah array, Blackrock Microsystems) into the ventral-precentral gyrus of a 45-year man with severe dysarthria (participant “T15”) living with ALS. As he attempted to speak, neural features were decoded by a deep neural network and parallel paralinguistic decoders, designed to synthesize speech continuously and in real-time, with a 10ms latency. RESULTS: The speech prosthesis recreated its synthetic speech target (r=0.90) in real-time at T15’s natural attempted speaking pace (30 words per minute). During real-time speech generation, the device could readily differentiate which word of a sentence was being emphasized (95.5% accuracy), and whether the sentence was phrased as a question (89.8% accuracy). Synthesized speech was readily modified based on these factors. Finally, we demonstrate the ability for our system to emulate his pre-ALS voice. CONCLUSIONS: We demonstrate preliminary results for real-time speech neuroprosthesis designed to restore the voice of a man with severe dysarthria, including phonemic as well as intonation content. Further work will focus on further improving the quality of the synthesized speech and quantifying the ability for listeners to understand it.