Development and evaluation of a prediction model for adult ICU hemorrhage using only continuous cardiorespiratory data

<i>Objective.</i>Develop and evaluate whether a model trained to detect the physiological signature of hemorrhage in intensive care unit (ICU) patients generalizes to other cohorts.<i>Approach.</i>We collected cardiorespiratory monitoring data and packed red blood cell administration data from consecutive adult admissions in one development and three evaluation ICU cohorts. We defined hemorrhage as three or more transfusions within 24 h. We trained a penalized logistic regression model to predict hemorrhage within 8 h and externally evaluated the predictions.<i>Main results.</i>The evaluation ICU cohorts comprised more than 6M q15 min observations. The cross-validated area under the receiver operating characteristic (AUC) in the development cohort was 0.706 (141 event admissions, 95% confidence intervals (CIs): 0.656-0.757) and 0.712 (968 event admissions, 95% CI: 0.693-0.726) for 17 591 medical and surgical ICU patients in the combination of 3 evaluation cohorts. The calibration slope of the hemorrhage model was close to unity (1.041, 95%CI: 0.956-1.127). Predicted risk increased significantly in the 8 h preceding clinical recognition of bleeding. There was no evidence of model performance drifting over time. There was evidence for lower performance for patients over 75 (20% lower than patients 18-44), among patients at University of Pittsburgh (Pitt) (24% lower than MIMIC III), Black patients (11% lower than White patients), and females (12% lower than males). The shock index also had reduced performance at Pitt, for female patients, and for patients over 75, though not for Black patients. The hemorrhage score had a higher net benefit than the shock index.<i>Significance.</i>Patients in ICUs have an increased risk for bleeding due to their chronic and acute illness, and earlier bleeding identification leads to better outcomes. A risk model for hemorrhage based only on continuous cardiorespiratory data has clinically relevant predictive performance that generalizes across three cohorts with different monitoring devices and electronic health record systems.