Explainable machine learning predicts mortality in critically ill patients with nonvariceal upper gastrointestinal bleeding: a MIMIC-IV study with external validation

Non-variceal upper gastrointestinal bleeding (NVUGIB) poses significant mortality risk in critically ill patients, necessitating accurate early prognostic assessment for timely interventions. Recent advances in machine learning have demonstrated potential to significantly improve predictive performance compared to conventional clinical scores. Accordingly, this study aims to establish a machine learning model named NVUPreM to predict 30-day NVUGIB mortality and validate its superiority over traditional scoring systems. This retrospective study derived data from the Medical Information Mart for Intensive Care IV (n = 2912) and the eICU Collaborative Research Database (n = 7,742) for model development and external validation, respectively. Predictors were selected via least absolute shrinkage and selection operator regression to minimize multicollinearity. Thirty-six machine learning algorithms were evaluated using tenfold cross-validation. The optimal model (NVUPreM) was compared against eight clinical scoring systems (AIMS65, Charlson, GBS, GCS, Admission-Rockall, SAPS II, SOFA) using the area under the receiver operating characteristic curve (AUC), calibration, decision curve analysis, and SHapley Additive exPlanations (SHAP) for interpretability. Among 2,329 eligible NVUGIB patients, 489 (21.0%) died within 30 days. The NVUPreM model demonstrated good discrimination (AUC = 0.876, [95% CI 0.846–0.907]) and sensitivity (0.86), showing consistently strong predictive performance among all models. In internal validation, the NVUPreM model outperformed all clinical scores as evidenced by superior AUCs (AIMS65: AUC = 0.693; Charlson: AUC = 0.636; GBS: AUC = 0.575; GCS: AUC = 0.707; NVUPreM: AUC = 0.876; Admission-Rockall: AUC = 0.633; SAPS II: AUC = 0.777; SOFA: AUC = 0.665), decision curve analysis and calibration curve. External validation in eICU confirmed the robustness of the NVUPreM model in terms of discrimination (AUC = 0.82, [95% CI 0.803–0.837]), calibration, and clinical application. The interpretability analysis illustrated the relative importance of individual predictors and their directional associations with 30-day mortality risk. The NVUPreM model significantly outperforms existing clinical scores in predicting 30-day NVUGIB mortality, offering both accuracy and interpretability, which could assist clinicians in early high-risk patient identification and personalized intervention.