S3107 Optimizing Dyspepsia Referrals: A Machine Learning Approach to Alarm Feature Detection

Introduction: Dyspepsia is a common condition addressed in primary care visits. Yet, despite established clinical guidelines emphasizing alarm features and demographic parameters for endoscopic evaluations, there is a significant amount of variability in patient referrals. This ultimately leads to 2 critical points: over-referral of low-risk patients to an already inundated system, and potential missed malignancies (approximately 11% of neoplasms are overlooked). Artificial intelligence (AI) has a role to parse through structured electronic health record (EHR) data and act as a tool to aid physicians in better sorting individuals for endoscopy. We created an AI model to identify dyspepsia patients requiring endoscopy, aiming to reduce unnecessary referrals while maintaining diagnostic safety. Methods: We conducted a retrospective cohort study (2021-2024) of 293 patients with dyspepsia from multi-center primary care clinics and its EHR. Using Bio_ClinicalBERT, a natural language process (NLP) model, we extracted search terms such as “melena” and “unintentional weight loss”. Utilizing the XGBoost machine learning model, we combined these terms with quantifiable trends such as “hemoglobin decline >2 g/dL”, “% weight loss” and “age” to create an Alarm Score. Primary endpoints include sensitivity for significant endoscopic findings, and net benefit analysis compared to actual practice. Results: Our AI model exhibited an improvement compared to clinical practice, achieving an area under the receiving operator curve of 0.83 (95% confidence interval 0.81-0.85) compared to 0.76 for physician practice. Additionally, it missed 0/13 malignancies, compared to 2/13 missed in usual care (99% sensitivity). Implementation of our model would have resulted in 28% fewer overall referrals to endoscopy, from 29% to 21%. The strongest predictive factors include NLP-identified weight loss documentation (odds ratio [OR] 8.2), melena (OR 7.3), and men over 60 with NSAID use (OR 9.1). Average cost savings per procedure were estimated to be $1,200. Conclusion: This study presents a new AI model to triage patients with dyspepsia for endoscopic referrals. Our model’s high sensitivity addresses an important safety concern, and its proposed reduction in unnecessary referrals presents significant cost savings and improving physician’s workloads. A limitation is the model’s reliance on documentation quality where it may miss some colloquial terms. Future direction would be to implement clinical decision tools like our model in real time to enhance clinical judgement.