Simulation-Based Optimal Insulin Infusion Control with Reinforcement Learning

The control of insulin infusion pump has been receiving attention from various sectors since artificial intelligence (AI) has been successfully applied to many optimal control problems. Data-driven control has become more popular with help of AI without dynamic model structure, but a white-box control would be more efficient if there exist an identified model structure. In case of insulin pump control design, traditional mathematical glucose-insulin models will be very useful to find an optimal insulin infusion profile by applying AI technique. The objective of insulin pump control in this study is to minimize the total amount of insulin dosage while maintaining the plasma glucose concentration level within a healthy range by controlling the profile of insulin infusion rate. We use Reinforcement learning (RL) to address this dynamic glucose-insulin interaction problem with a goal of achieving minimum insulin dosage. RL in this study finds an optimal insulin infusion profile with minimum dosage that stabilizes a plasma glucose concentration from an initial hyperglycemic state to a basal level in a reasonable amount of time. Since diabetes is known as a chronic disease, minimal insulin dosage control may be important to decrease potential insulin resistance and play an important role in the long-term treatment of diabetes patients. We provide numerical examples of glucose-insulin models and verify our proposed RL insulin infusion profile by comparing it with two insulin infusion programs from Fisher's study.