Deployable Machine Learning on MIMIC-IV: Leakage-Safe Prediction and Calibration for Incident Diabetes

We present a leakage-safe, deployment-oriented pipeline for predicting incident diabetes within 365 days of an index hospital admission, using the MIMIC-IV (Medical Information Mart for Intensive Care) hospital module. The process enforces temporal and patient-level guards, separates model-specific preprocessing, and evaluates models with a calibration-first lens: reliability diagrams and Brier score, alongside discrimination (ROC/PR) and decision-analytic utility via decision curves. Two standard baselines—logistic regression and random forest—are tuned with inner cross-validation and post hoc calibrated (Platt or isotonic) tools. On the held-out test set, calibrated logistic regression attains modest yet consistent performance (e.g., AUROC≈0.70, AUPRC≈0.31) and competitive net benefit across clinically plausible thresholds; calibrated random forest is lower. Global explanations (permutation importance; SHAP for logistic) yield clinically coherent reliance patterns (age, length of stay, sex, ethnicity). The primary contribution is procedural and implementation-oriented: a transparent, reproducible, leak-proof pipeline that surfaces calibrated risks, threshold-aware decision support, and auditability artifacts suitable for clinical review and site adaptation. We outline a practical path to external validation and lightweight site-specific recalibration, positioning the work as a reference implementation for reliable EHR risk stratification rather than an algorithmic innovation.