Development of a Machine Learning-Based Triage Score for Medication-Related Osteonecrosis of the Jaw in Osteoporosis Patients Undergoing Tooth Extraction

Background/Objectives: Medication-related osteonecrosis of the jaw (MRONJ) is a serious complication in osteoporosis patients undergoing tooth extraction. This study aimed to develop and evaluate an interpretable, machine learning–derived triage score for rapid risk stratification at the initial dental visit. Methods: This retrospective study included 850 osteoporosis patients (443 MRONJ, 407 controls) in the derivation cohort and 559 independent multicenter MRONJ cases for external evaluation. A reference random forest model identified a hierarchical feature structure, which was translated into an additive integer-weighted scoring system through systematic hyperparameter optimization. Structural tipping points were identified using isotonic regression and first discrete derivative analysis. Internal performance was further characterized by sensitivity, specificity, PPV, NPV, calibration slope and intercept, the Hosmer–Lemeshow test, decision curve analysis, bootstrap optimism correction, and subgroup analyses. External evaluation assessed three-tier distribution concordance and case capture rates with non-inferiority testing. Results: The reference random forest achieved an AUC of 0.792. The final MRONJ triage score (range 0–17) incorporated six binary predictors with mutually exclusive drug route categories. The triage score preserved discriminative performance (AUC 0.772; ΔAUC = 0.020; p = 0.149). Two tipping points at scores 7 and 14 defined three risk tiers: low (0–6; 20.9%), moderate (7–13; 55.3%), and high (≥14; 83.5%). At the moderate-risk threshold (≥7), the score achieved sensitivity 90.3% (95% CI 87.2–92.7%) and specificity 45.0% (40.2–49.8%); at the high-risk threshold (≥14), specificity rose to 91.4% and PPV to 83.1%. Calibration was adequate (slope 0.994; intercept 0.0006; Hosmer–Lemeshow p = 0.381), and decision curve analysis demonstrated higher net benefit than reference strategies across all clinically relevant threshold probabilities. The bootstrap optimism-corrected AUC was 0.778, and discriminative performance remained stable across age, route, duration, and site subgroups (AUC range 0.70–0.79). In the external cohort, the case capture rate at the ≥7 threshold was non-inferior (83.4% vs. 88.0%; Δ = −4.6%; margin −10%). Conclusions: The MRONJ triage score demonstrated stable discrimination and reproducible case capture in an independent multicenter cohort. By relying on six variables obtainable at the initial dental visit, this framework may have the potential to reduce unnecessary tertiary referrals and support safer clinical decision-making, although this benefit was not directly demonstrated and requires confirmation in prospective implementation studies.