Predicting Thirty Day Hospital Readmission in Patients With Diabetes Using Explainable Machine Learning

Abstract Background Hospital readmission within 30 days of discharge is an important healthcare outcome. Machine-learning models may help estimate readmission risk; however, their performance, outcome definitions, and interpretability must be carefully evaluated before potential clinical application. Objective To evaluate Logistic Regression and Random Forest models for predicting recorded 30-day hospital readmission among encounters in the publicly available UCI Diabetes 130-US Hospitals dataset. Methods This retrospective secondary-data study used 101,766 hospital encounters recorded between 1999 and 2008. Encounters labelled < 30 in the original dataset were classified as positive cases, whereas those labelled > 30 or NO were classified as negative cases. This outcome definition reflects recorded readmission timing and does not distinguish planned from unplanned readmissions. Logistic Regression and Random Forest models were evaluated using a stratified, patient-grouped training–test partition to prevent encounters belonging to the same patient from appearing in both partitions. Model performance was assessed using the area under the receiver operating characteristic curve (AUROC), average precision, and classification metrics. Model interpretation was considered, but reliable feature-level explanations were not established. Results The held-out test set contained 20,353 encounters, including 2,334 positive cases (11.47%). Logistic Regression achieved an AUROC of 0.648 and average precision of 0.200, while Random Forest achieved an AUROC of 0.659 and average precision of 0.213. At their default classification rules, Logistic Regression identified 41 positive cases, yielding a sensitivity of 1.76% and specificity of 99.55%. Random Forest classified all test encounters as negative, yielding a sensitivity of 0% and specificity of 100%. Despite accuracies of 88.33% and 88.53%, respectively, both models showed substantial limitations in identifying positive cases. Reliable predictor rankings and SHAP explanations were not established; therefore, the relative contributions of individual predictors remain uncertain. Conclusions Both models demonstrated modest discrimination and inadequate positive-case detection under their default classification rules. Accuracy alone was insufficient to demonstrate useful readmission prediction in this imbalanced dataset. The historical, diabetes-specific population and the source outcome definition limit generalizability. Neither model is ready for clinical deployment based on these results. Future research should validate model explanations, assess calibration and subgroup performance, select decision thresholds using training or validation data, evaluate clinical utility, and conduct external validation using contemporary clinical datasets.