Closed-Loop Neuromorphic Deep Brain Stimulation using Deep Spiking Q-Networks

Current open-loop deep brain stimulation (DBS) implants continuously apply electrical current to reduce motor symptoms in patients with Parkinson’s disease (PD). However, neural dynamics are patient-specific, and open-loop DBS systems are energy-inefficient as they can provide ineffective and unnecessary stimulus. Closed-loop DBS systems offer a more efficient and adaptive approach to delivering DBS. Advances in simulating biomarker responses across cortex-basal ganglia-thalamus (CBGT) networks has accelerated closed-loop DBS development. While deep learning shows promise for optimizing closed-loop stimulation, its high computational demands challenge the battery life of implanted DBS devices. Spiking neural networks (SNNs) offer an energy-efficient alternative to traditional neural networks. They benefit from the ability to take advantage of sparse activations of neurons and to transmit information as spikes. We propose a test bench for DBS parameter optimization by introducing a rat model of the CBGT network in a reinforcement learning (RL) environment and train a deep spiking Q network (DSQN) to validate an end-to-end spike model. This marks a step towards the first closed-loop benchmark with end-to-end spiking, from sensory inputs, to the model, to the stimulus outputs.