High accuracy of large language models in delivering recommendations for Barrett’s esophagus management using real-world multilingual data—proof of concept study

Barrett's esophagus (BE) is a premalignant condition. Societal guidelines standardize its management to prevent progression to adenocarcinoma, yet adherence varies substantially between gastroenterologists. This study aimed to assess and compare the accuracy of large language models in delivering guideline-based recommendations for managing BE. We evaluated 107 cases of patients with BE, each case included all available esophagogastroscopy and pathology reports. Two experts provided recommendations according to the European Society of Gastrointestinal Endoscopy, focusing on three aspects: (i) pathology confirmation by a second pathologist; (ii) next action-and (iii) the interval until next endoscopy. Gemini 2.0 and o3-mini-high were evaluated. Training with prompt engineering and testing were performed on 23 and 84 cases respectively. Gold standard was defined as the consensus recommendations provided by the experts. Accuracies were calculated using bootstrapping, and McNemar's test was employed for large language models comparison. Both large language models demonstrated high to near perfect accuracy across all three fields as compared to expert recommendations, without significant differences between the two models for all parameters. Agreement between the two large language models was high in all three fields. Large language models demonstrated high accuracy in assessing real-world presentations of BE and providing guideline-based recommendations for patient management.