Deep Reinforcement Learning for Automated Insulin Delivery Systems: Algorithms, Applications, and Prospects

Advances in continuous glucose monitoring (CGM) technologies and wearable devices are enabling the enhancement of automated insulin delivery systems (AIDs) towards fully automated closed-loop systems, aiming to achieve secure, personalized, and optimal blood glucose concentration (BGC) management for individuals with diabetes. While model predictive control provides a flexible framework for developing AIDs control algorithms, models that capture inter- and intra-patient variability and perturbation uncertainty are needed for accurate and effective regulation of BGC. Advances in artificial intelligence present new opportunities for developing data-driven, fully closed-loop AIDs. Among them, deep reinforcement learning (DRL) has attracted much attention due to its potential resistance to perturbations. To this end, this paper conducts a literature review on DRL-based BGC control algorithms for AIDs. First, this paper systematically analyzes the benefits of utilizing DRL algorithms in AIDs. Then, a comprehensive review of various DRL techniques and extensions that have been proposed to address challenges arising from their integration with AIDs, including considerations related to low sample availability, personalization, and security are discussed. Additionally, the paper provides an application-oriented investigation of DRL-based AIDs control algorithms, emphasizing significant challenges in practical implementations. Finally, the paper discusses solutions to relevant BGC control problems, outlines prospects for practical applications, and suggests future research directions.