Identifying incident medication‐related osteonecrosis of the jaw and antiresorptive drug holidays using natural language processing on clinical notes in men and women prescribed bisphosphonates or denosumab for fracture prevention

OBJECTIVE: Medication-related osteonecrosis of the jaw (MRONJ) is a rare complication of antiresorptive therapy for osteoporosis, with risk potentially influenced by "drug holidays" around dentoalveolar procedures. MRONJ risk is higher in rheumatic diseases. Epidemiologic studies are limited by poor administrative code performance for MRONJ identification and lack of structured pharmacy data capturing drug holidays. We developed and validated a natural language processing (NLP) algorithm to classify MRONJ and detect drug holidays from free-text clinical notes in electronic health records (EHR). METHODS: Using U.S. Veterans Health Administration EHR, we identified adults ≥50 years old with ≥1 filled prescription for a bisphosphonate or denosumab for fracture prevention (10/1/1999-12/31/2022). An NLP codebook defined 11 targets with 16 attributes. Two annotators independently labeled notes to create a reference set (n=870; 200 held out for validation) and an independent validation set (n=100) for performance assessment (precision, recall, F-measure). A decision tree-based machine-learning model used NLP-extracted features to classify MRONJ. Drug holiday was an NLP target directly. RESULTS: The reference set included 6,421 annotations informing a curated vocabulary of >1,350 terms. For MRONJ classification excluding unclassifiable outputs across all validation notes, precision was 1.00 (95% confidence interval [CI] 0.86-1.00), recall 0.96 (0.80-1.00), and F-Measure 0.95 (0.88-1.00). For drug holiday detection in the held-out validation set, precision was 0.90 (0.83-0.95), recall 0.98 (0.93-1.00), and F-Measure 0.94. CONCLUSION: We developed an NLP algorithm that is sufficiently accurate in classifying MRONJ and detecting drug holidays to be a promising tool for future studies, including in persons with rheumatic diseases.