Importance Sampling to Learn Vasopressor Dosage to Optimize Patient Mortality in an Interpretable Manner

Sepsis is a life-threatening organ dysfunction resulting from an improperly compensated bodily response to infection. There is a high urgency among clinicians to develop a set of realtime explainable treatment guidelines and tools to address the high mortality rate of sepsis patients. We present a reinforcement learning approach for vasopressor drug dosage in intensive care unit sepsis patients to achieve a better-than-expert treatment policy. We preserved interpretability with a prototype learning layer and learn actions in an off-policy manner with importance sampling. We evaluated our design on the MIMIC-IV deidentified electronic health record dataset with an 80%-20% training-validation split for 300 epochs. For patient trajectories where expert actions agreed with over 80% of model policy actions, we achieved a mortality rate of approximately 35%, outperforming the expert policy mortality rate of 38%. Where expert actions agreed with the model policy for less than 80% of states, patient mortality reached 92%. This work indicates the model policy outperformed the expert policy by a significant margin while preserving interpretability. Future directions include improved data cleaning and extraction techniques, more tailored patient feature selection, exploring solutions to address limited trajectory exploration in training data, and improved evaluation metrics as well as evaluation on other clinical datasets.