Brain Chips in the Rough: Computational Modeling of Brain Stimulation by an Intercept and Replace Machine Learning Model

Inthis study, we proposed a closed-loop transformer neural network decoder-based signal generation method to record, decode, and stimulate the brain to intercept a previously functional brain area. The dataset consists of the auditory cortex output of computational modeling of the human auditory system, following voice input. The system is comprised of three phases. In Phase I artificial brain signal was generated based on a generative sequence-to-sequence machine learning model. In Phase II the cortical area downstream was stimulated with the generated brain signal, effectively replacing the upstream cortical area on which the machine learning model is based. Finally, in Phase III, incremental learning is applied to fine-tune the model to unlock differences between what is being recorded by our electrodes and the hidden neural patterns of the brain. As an evaluation, we computed RMSE and similarity index between the generated stimulation output and inputted original computational model output without any interception. The generative regression model using our proposed method shows an average RMSE = 0.009845, and similarity index (SI) = 0.9416 across the models without an word embedding layer, and SI = 0.855283 and RMSE = 0.033068 for the model with a word embedding layer. This could be evidence to support that the Transformer-Decoder model has extracted distinguishable features during heard speech.