Development and validation of a prior-to-admission medication list risk scoring tool

Abstract Purpose To develop, validate, and implement an admission predictive model that estimates the likelihood and expected number of changes to the prior-to-admission (PTA) medication list, to help triage pharmacy-led medication histories. Methods We performed a retrospective study of adult admissions at a large academic medical center (2019-2025). Using electronic health record (EHR) data available at admission, we built a 2-stage (“hurdle”) gradient-boosted tree model: Stage 1 predicts whether any PTA list changes will occur; stage 2 predicts the number of changes among encounters with 1 or more changes. The primary outcome was a weighted count of PTA list changes (additions, deletions, and modifications), with triple weight for prespecified high-risk medication classes. We evaluated discrimination, calibration, count error, and capacity-aware ranking (eg, precision at 25%, capture at 25%). Fairness analyses examined performance across age, sex at birth, and race. Results Among 402,023 encounters, 33.1% had no PTA medication list discrepancies. Stage 1 achieved an area under the curve of 0.816 and an area under the precision-recall curve of 0.886; probability calibration was acceptable (log loss, 0.479; Brier score, 0.15). For stage 2, the mean absolute error was 3.6 and the root mean square error (RMSE) was 5.2; log1p-RMSE was 0.63. Under a 25% review capacity, precision was 90.6% and capture was 57.4% (normalized discounted cumulative gain was 0.762). Conclusion A 2-stage, calibration-aware model using admission-available EHR data can effectively prioritize patients who are most likely to need pharmacist-obtained medication histories, supporting more efficient use of scarce pharmacy resources.