Sepsis Early Warning: External Validation of a Physiology-Informed Machine Learning Model Across Hospital Systems

Background: Sepsis remains a leading cause of in-hospital mortality worldwide. Delayed recognition significantly worsens clinical outcomes, and existing scoring tools are typically applied only after clinical suspicion has already been raised. Methods: This study developed a physiology-informed LightGBM machine learning model to predict sepsis onset up to 6 hours in advance, using ICU data from two independent hospital systems (PhysioNet/Computing in Cardiology Challenge 2019). Set A (n = 20,336) was used for model development and internal testing; Set B (n = 20,000), from a different hospital system, was reserved exclusively for external validation without retuning. Features were engineered causally, using only information available up to each hour, and the alert threshold was selected strictly on validation data before any evaluation on Set B. Confidence intervals were computed using patient-level bootstrap resampling (100 iterations). Results: The model achieved an AUROC of 0.828 (95% CI: 0.806–0.851) on internal testing and 0.776 (95% CI: 0.762–0.789) on external validation. At the validation-selected threshold, 42.8% of sepsis patients in the external cohort were flagged before clinical onset (n = 1,142), with a 6.8% false-alarm rate among non-sepsis patients. Performance was consistent across sex and age subgroups (AUROC 0.76–0.78). Conclusions: SHAP analysis showed the model relies substantially on ICU length of stay, a feature that may partly reflect dataset labelling rather than physiology, alongside physiologically coherent markers including oxygenation demand, inflammatory and renal markers, and an engineered shock index. The model demonstrates reasonable generalisation across distinct hospital systems, though calibration and label-driven dependencies warrant further work before any clinical use. Code, an interactive demonstration, and full results are publicly available.