A biomimetic deep encoder for auditory restoration by cortical stimulation

Cochlear and auditory brainstem implants are the primary treatments for profound deafness. However, the inherently low resolution of electrical stimulation used by these devices limits information transfer to a small number of independent stimulation channels and results in perceptual difficulties. An alternative approach is to target the auditory cortex, a brain region large enough to accommodate hundreds of stimulation channels and whose stimulation evokes sound percepts. Although modern brain interfaces offer large channel counts and spatial resolution, developing encoding algorithms capable of accurately transmitting interpretable sound information remains challenging. Here we introduce Braincodec, a deep network that optimizes both sound information transmission and the alignment of stimulation patterns with the spatial, temporal, and high-level properties of the auditory cortex neural code. Braincodec operates online and requires only a few hundred channels to transmit sufficient information for the reconstruction of natural sounds with high fidelity, a performance inaccessible to cochlear implant encoders. Moreover, mice equipped with a Braincodec-driven auditory cortical prosthesis matched the accuracy of non-implanted mice in discriminating real sounds. Braincodec therefore enables the development of high-throughput cortical neuroprosthetics with the potential to overcome the limitations of current auditory implants and provides a generic framework applicable to other sensory modalities.