Comparative Evaluation of Machine Learning Models Using Structured and Unstructured Clinical Data for Predicting Unplanned General Medicine Readmissions in a Tertiary Hospital in Australia

Background: Unplanned 30-day hospital readmissions, a key healthcare quality metric, are common and costly. Prediction models built on structured data often perform modestly, and the added value of unstructured clinical notes remains unclear. Methods: This retrospective cohort study included 4135 general medicine admissions to a tertiary Australian hospital between July 2022 and June 2023. Structured predictors included demographics, comorbidities, frailty, prior healthcare utilisation, length-of-stay, inflammatory markers, socioeconomic indicators, and lifestyle factors. We developed deep learning models using structured data alone, unstructured text alone, and a combined multimodal architecture integrating both modalities. For benchmarking, multiple classical machine learning models trained on structured features were evaluated using identical data splits, including logistic regression, XGBoost, random forest, gradient boosting, extra trees, and HistGradient Boosting. Model performance was assessed on a hold-out test set using ROC-AUC, accuracy, precision, recall, and F1-score. Results: Unplanned readmissions occurred in 24.3% of admissions. Among classical machine learning models, logistic regression achieved the highest discrimination (ROC-AUC 0.64), with no substantial improvement observed from ensemble methods. Structured-only deep learning achieved ROC-AUC 0.62. Unstructured text-only and multimodal models achieved ROC-AUCs of 0.52 and 0.58, respectively. Although overall discrimination of the multimodal model was lower than structured-only performance, it demonstrated improved sensitivity and F1-score for identifying patients who were readmitted. Prior hospitalisations, emergency department visits, and comorbidity burden were the strongest predictors. Conclusions: Structured EMR variables remain the main drivers of 30-day readmission risk. More complex classical machine learning models did not outperform logistic regression, and incorporating unstructured clinical text provided only modest improvement in identifying high-risk patients without enhancing overall discrimination.