Leveraging natural language processing to assess follow-up patterns in glaucoma care

PURPOSE: To develop a natural language processing (NLP) algorithm that identifies follow-up recommendations from the free text of clinical glaucoma clinic notes, and to determine the proportion of patients who lapse beyond recommended follow-up intervals. DESIGN: Retrospective cohort study. PARTICIPANTS: Adult glaucoma patients treated from January 2016 through December 2022 at our academic ophthalmology practice. METHODS: Structured demographic and diagnosis data were extracted alongside the free text of clinical notes. An NLP algorithm was developed to identify recommended follow-up intervals from note free text. Lapses in care were defined as absence of a return visit within 125% of the recommended follow-up interval. Generalized estimating equations with a Poisson distribution and log link were used to identify risk factors for lapsing in care. Results are reported as adjusted risk ratios (aRRs) with 95% confidence intervals (CIs). MAIN OUTCOME MEASURES: NLP algorithm performance and proportion of patients with an identified lapse in care. RESULTS: In total, 221,378 clinical notes were assessed, representing 66,228 visits by 6452 unique glaucoma patients. The NLP algorithm identified a follow-up interval for 47,084 of 66,228 (71.1%) patient visits and demonstrated high performance compared to gold-standard chart review (F1 score 0.905, precision 0.948, recall 0.867). The proportion of visits that exceeded 125% of the preceding recommended follow-up interval was 20.0% [interquartile range (IQR): 10.0-33.3]. The median excess duration of the lapsed interval compared to recommended follow-up was 52.9 days (IQR: 22.0-111.5 days). The proportion of patients with any event of lapsing in follow-up was 73.7% (4754/6452). Risk factors for having at least one lapse in care included Black race (aRR=1.29, 95%CI: 1.22-1.37, compared to White race) and Hispanic ethnicity (aRR=1.53, 95%CI: 1.26-1.87), with lower risk for older age (aRR=0.91, 95%CI: 0.86-0.97, age 71-80 years compared to ≤60 years). Among the 4,754 patients with at least one lapse in care, 1,183 (24.9%) never returned after lapsing. CONCLUSIONS: NLP can reliably detect lapses in glaucoma care using individualized follow-up intervals. Identifying lapses in care may facilitate referrals to resources to mitigate risk of loss to follow-up.