A Hybrid Closed-Loop Blood Glucose Control Algorithm with a Safety Limiter Based on Deep Reinforcement Learning and Model Predictive Control

Due to the complexity of blood glucose dynamics and the high variability of the physiological structure of diabetic patients, implementing a safe and effective insulin dosage control algorithm to keep the blood glucose of diabetic patients within the normal range (70-180 mg/dL) is currently a challenging task in the field of diabetes treatment. Deep reinforcement learning (DRL) has proven its potential in diabetes treatment in previous work, thanks to its strong advantages in solving complex dynamic and uncertain problems. It can address the challenges faced by traditional control algorithms, such as the need for patients to manually estimate carbohydrate intake before meals, the requirement to establish complex dynamic models, and the need for professional prior knowledge. However, reinforcement learning is essentially a highly exploratory trial-and-error learning strategy, which is contrary to the high-safety requirements of clinical practice. Therefore, achieving safer control has always been a major challenge for the clinical application of DRL. This paper addresses this challenge by combining the advantages of DRL and the traditional control algorithm-model predictive control (MPC). Specifically, by using the blood glucose and insulin data generated during the interaction between DRL and patients in the learning process to learn a blood glucose prediction model, the problem of MPC needing to establish a patient's blood glucose dynamic model is solved. Then, MPC is used for forward-looking prediction and simulation of blood glucose, and a safety controller is introduced to avoid unsafe actions, thus restricting DRL control to a safer range. Experiments on the UVA/Padova glucose kinetics simulator approved by the US Food and Drug Administration (FDA) show that the time proportion of adult patients within the healthy blood glucose range under the control of the model proposed in this paper reaches 72.51%, an increase of 2.54% compared with the baseline model, and the proportion of severe hyperglycemia and hypoglycemia events is not increased, taking an important step towards the safe control of blood glucose.