Personalized adaptive suppression of pathological oscillations via entropy-regularized reinforcement learning

A nonlinear disturbance rejection controller combined with entropy-regularized reinforcement learning (NDRC-ERRL) is developed to suppress pathological oscillations in a delayed subthalamic nucleus-globus pallidus external (STN-GPe) firing-rate model. The controller combines an unknown-input observer with nonlinear state-error feedback, while a bounded soft actor-critic policy adjusts two observer gains and two feedback gains every 0.02 s. The policy is trained under randomized coupling and disturbance multipliers and is deployed using deterministic gain actions within prescribed safety bounds. The method was evaluated over five-second simulations for controller activation at 0.2, 0.5, and 1.0 s and under a fixed ON–OFF schedule. Conventional active disturbance rejection control and sliding mode control were used as benchmarks. Performance was assessed using time-weighted RMSE, MAE, and MSE over active-control intervals. Across the four scenarios, NDRC-ERRL achieved a mean RMSE of 0.4143 spikes/s, compared with 0.4816 spikes/s for C-ADRC and 1.5000 spikes/s for SMC, corresponding to reductions of 14.0% and 72.4%, respectively. The proposed controller also achieved the smallest MAE and MSE in every tested scenario. These in silico results demonstrate improved tracking and repeatable recovery after controller reactivation within the considered delayed STN-GPe model and motivate further validation using more comprehensive neural models and experimental recordings.