Threshold-Optimized Electronic Health Record-Based Machine Learning for Predicting 1-Year Acute Care Use in Adults with Diabetes at an Urban Health Care System

Background/Objectives: Acute care use (ACU)—emergency department visits, inpatient hospitalizations, and observation stays—drives morbidity and costs among adults with diabetes. We developed and evaluated machine-learning models to predict 1-year ACU risk using electronic health record (EHR) data and neighborhood-level data. Methods: We performed a retrospective cohort study using de-identified EHR data from a large urban academic health center, including adults (≥18 years) with diabetes (N = 23,052). The index date was defined as one year before each patient’s last encounter, and ACU was assessed during the subsequent year. We modeled 180 predictors spanning demographics, Area Deprivation Index (ADI), prior healthcare utilization, vitals/BMI, comorbidities, medications, and laboratory results. Decision tree and gradient-boosted models (XGBoost, LightGBM, CatBoost) were tuned with Optuna using 8-fold stratified cross-validation, optimizing area under the receiver operating characteristic curve (AUC). To improve class-balanced classification performance under outcome imbalance, we selected post hoc probability thresholds that maximized Macro F1 and quantified interpretability with permutation feature importance. Results: ACU occurred in 30.53% of patients (7039/23,052). Boosted models achieved AUC ≈ 0.78, with LightGBM performing best (AUC = 0.7839). Macro F1–optimized thresholds (<0.5; typically 0.375–0.40) improved class-balanced performance versus a 0.5 cutoff. Across boosted models, prior utilization features dominated, followed by discharge-related factors and neighborhood deprivation; comorbidities and laboratory results contributed. Conclusions: In this single urban academic health-system cohort of adults with diabetes, EHRbased boosted models demonstrated moderate discrimination for predicting 1-year ACU and identified interpretable predictive signals. Threshold optimization improved class-balanced statistical performance. Prior utilization, care transitions, and neighborhood deprivation emerged as dominant predictive features. External and temporal validation are needed before broader application.