Development and external validation of a machine learning model for predicting post-CABG acute kidney injury: insights from MIMIC database and real-world cardiac surgery cohort

OBJECTIVE: Acute kidney injury (AKI) is a severe complication following coronary artery bypass grafting(CABG) While machine learning models trained on large-scale intensive care unit (ICU) databases are increasingly prevalent, their ability to generalize to specialized surgical cohorts in real-world settings remains poorly characterized. This study aimed to develop a post-CABG AKI prediction model using general ICU data and rigorously evaluate its transferability to an independent cardiac surgery center. METHODS: A retrospective dual-center study was conducted. The development cohort comprised 6,347 post-CABG patients from the MIMIC database. An independent external validation cohort included 534 patients from the Affiliated Hospital of Xuzhou Medical University (AHXY). Least absolute shrinkage and selection operator regression was employed for feature selection. Nine ML algorithms were benchmarked, with Random Forest selected as the optimal model. Performance was assessed using the area under the precision-recall curve (PR-AUC), area under the ROC curve (AUC), and decision curve analysis. SHapley Additive exPlanations (SHAP) were used to interpret feature contributions. RESULTS: In the MIMIC internal testing set, the Random Forest model demonstrated moderate discrimination, achieving a PR-AUC of 0.888 (95% CI: 0.871-0.904) and an AUC of 0.709. However, upon deployment in the external AHXY cohort, the model exhibited significant performance attenuation, with the PR-AUC dropping to 0.309 and the AUC decreasing to 0.548. This near-random predictive performance indicates a lack of direct real-world generalizability without further adaptation. SHAP analysis identified body weight, serum phosphate, and red cell distribution width as key predictors. The performance decay was attributed to significant population heterogeneity (e.g. BMI differences) and the specific pathophysiological milieu of cardiac surgery not fully captured by general ICU variables. CONCLUSION: General-purpose ICU models may face a substantial cross-domain transferability gap without local adaptation when applied to specialized sub-specialties like cardiac surgery across different demographic populations. Our findings suggest that direct deployment of such models carries clinical risks and highlight the imperative for localized model recalibration or transfer learning to account for domain-specific variations before clinical adoption.