Federated Reinforcement Learning for Therapeutic Interventions over ICUs with Noisy Labels

The proliferation of healthcare IoT devices and the resulting rich healthcare data sprout new possibilities for intelligent healthcare applications. Patients in intensive care units (ICUs) rely on various networked gadgets to continuously monitor their health and manage critical situations. Among the common therapeutic interventions in ICUs, invasive mechanical ventilation and injecting sedatives during ventilation play crucial roles in maintaining respiratory function and enhancing patient care. While existing therapeutic interventions largely depend on experience and intuition, we propose a federated inverse reinforcement learning framework, termed FERRY, which automatically and intelligently learns optimal therapeutic intervention policies across networked ICUs while keeping raw data local. Specifically, our federated approach overcomes limitations in medical data privacy and facilitates collaboration; our proposed inverse reinforcement learning framework learns the variational posterior distribution from historical trajectories to handle the unknown reward. Additionally, we enhance our framework with distributionally robust optimization to ensure worst-case performance and adaptively filter out noisy data through joint loss learning. Extensive experiments on the real-world dataset demonstrate that FERRY improves the overall ventilation and sedation decision-making accuracy by 36.75% compared to other state-of-the-art baselines.