P2P-DML: Peer-to-Peer Distributed Machine Learning for Medical Deep Reinforcement Learning

Distributed Machine Learning (DML) offers significant potential to improve sample efficiency in Deep Reinforcement Learning (DRL), yet its application in this area is limited. This paper presents a novel DML algorithm for DRL, called Peer-to-Peer Distributed Machine Learning (P2P-DML). The algorithm consists of two phases: first, clients form a network ring where each trains the model for a set number of epochs before passing it to the next client, repeating this process multiple times. In the second phase, each client refines the model with additional training epochs for personalization. Inspired by Transfer Learning, P2P-DML is tailored for heterogeneous client environments in DRL. We evaluate this approach on the artificial pancreas problem, optimizing insulin delivery for type 1 diabetes patients. Our results show that P2P-DML achieves comparable performance while reducing training epochs by up to 82% compared to traditional bottom-up methods.