Impact of acute kidney injury labeling strategies in hospitalized patients: suggestion for a dual-alert clinical decision support system

Background: Operational variability in defining acute kidney injury (AKI) undermines the diagnostic reliability and impairs the generalizability of machine learning (ML) models.We evaluated whether the refined criteria based on the Kidney Disease: Improving Global Outcomes (KDIGO) guidelines enhance prognostic validity and predictive performance.Methods: Two conservative refinements to the KDIGO serum creatinine (SCr)-based criteria were proposed: 1) exclusion of transient SCr decreases from baseline estimation and 2) application of a minimum absolute SCr increase threshold.We generated 441 AKI labeling strategies by combining these refinements with baseline estimation methods.For each strategy, we calculated adjusted hazard ratios (aHRs) for adverse outcomes and assessed CatBoost model performance using the area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC).To assess real-world interpretability, ambiguous cases were reviewed.Results: Among 5,115 patients labeled by the standard KDIGO, 4,137 (80.9%) met the refined criteria, and 978 (19.1%) were considered ambiguous.The refined cohort had higher adverse event rates than the standard cohort (44.0% vs. 37.8%; aHR, 1.23).ML models trained with refined labels showed improved discrimination (AUROC, 0.935 vs. 0.926) and precision (AUPRC, 0.729 vs. 0.707) and reduced false-positive alerts by 27% with minimal loss in adverse outcome detection.Ambiguous cases often reflected transient measurement variability or frailty.Conclusion: This study presents a strategic AKI labeling approach based on KDIGO criteria and suggests a dual-alert system to distinguish refined AKI and ambiguous AKI cases.These approaches may enhance both predictive model performance and clinically meaningful detection.