Early Prediction of Carbapenem Resistance in Gram-Negative Organisms at Culture Ordering from Routinely Collected Hospital Data: Temporal and External Validation of ORDER-CARB

INTRODUCTION: Empirical therapy for severe suspected infection is selected before organism identification and antimicrobial susceptibility results are available. ORDER-CARB estimates, at culture ordering, the probability of subsequent recovery of a carbapenem-resistant Gram-negative organism using routinely collected structured data available before the index culture. METHODS: ORDER-CARB was developed using a large publicly available dataset, the Antibiotic Resistance Microbiology Dataset, with chronological model-fitting, calibration, and temporal-test splits. The frozen model was transported to the independent and also publicly available East Carolina University Health (ECUH) and University of Texas Southwestern (UTSW) cohorts, locally recalibrated using older data, and evaluated in future hold-outs. The primary analysis included microbiologically labeled urine, respiratory, and blood cultures that subsequently yielded at least one Gram-negative organism with interpretable carbapenem susceptibility. A secondary two-stage analysis evaluated all blood-culture episodes in the external cohorts. RESULTS: ORDER-CARB identified subsequent carbapenem-resistant Gram-negative isolates with an AUROC of 0.858 in internal temporal testing and of 0.797 at ECUH and 0.876 at UTSW in future external cohorts. Using locally selected cutoffs intended to identify approximately 9 of 10 resistant isolates and thereby prompt early review of whether empirical coverage against carbapenem-resistant Gram-negative organisms was warranted while susceptibility results were pending, sensitivity in the future cohorts was 91.6% at ECUH and 85.7% at UTSW. This approach would have prompted review in 55.4% and 27.7% of cultures, respectively, with positive predictive values of 5.7% and 9.4%. In the secondary analysis including all blood-culture episodes, irrespective of the subsequently identified organism, the model also showed high discrimination for carbapenem-resistant Gram-negative bloodstream infection, with AUROCs of 0.928 (95% CI 0.869-0.987) at ECUH and 0.967 (95% CI 0.909-1.000) at UTSW. CONCLUSIONS: ORDER-CARB retained useful ranking after chronological external transport, but the marked prevalence shift materially affected positive predictive value and review burden, and operating thresholds remained site-specific. The score should not replace susceptibility testing, severity assessment, or clinical judgment. Prospective evaluation is needed to determine whether it can support empiric treatment and stewardship decisions after local recalibration.