Dual-Gated Double Deep Q-Networks for Sepsis Resuscitation: An Offline Reinforcement Learning Framework for Hemodynamic Management in Intensive Care

Abstract Sepsis is a serious condition caused by an uncontrolled body response to infection, and it remains one of the primary cause of death in intensive care units worldwide. Giving the right amount of intravenous fluids and vasopressors at the right time is critical to keep blood flowing to vital organs. However, current clinical guidelines are broad and inconsistent. This leads to large differences in practice, where giving either too little treatment or too much fluid and medication can worsen organ failure and increase mortality. In this work, we formulate sepsis management over time as an offline Markov Decision Process using real clinical data from the MIMIC-IV v3.0 database. We introduce Dual-Gated Dueling Double Deep Q-Networks, a model-free reinforcement learning method developed to solve two major issues in critical care learning: severe overestimation of Q-values and the tendency of models to recommend dangerously high doses. The proposed network uses a learned confidence gating mechanism combined with an uncertainty gate to adjust smoothly between standard Double DQN targets and greedy updates. In addition, we add normalized advantage centering to keep the calculations mathematically stable and prevent risky treatments from distorting baseline state values, as well as a penalty term that encourages lower doses based on patient severity. We evaluated our model on separate patient test sets to ensure no data leakage occurred. The results show that our proposed model produces more stable treatment policies, reduces overestimation bias, and provides safer, more conservative dosage recommendations, suggesting conservative fluid and vasopressor amounts in 43.3% of patient states compared to roughly 21% in standard baseline models and 19% in actual clinician practice.