S569 Battle of the Bots: Real-World Comparison of Artificial Intelligence-Assisted Colonoscopy Systems Reveals Competitive Performance

Introduction: Artificial intelligence (AI)-assisted colonoscopy for polyp detection is increasingly prevalent in colonoscopy practices. Every AI system is built on a proprietary algorithm using a unique primary dataset yielding variable performance characteristics. As a result, AI systems inherently perform differently, and these performance differences may affect important patient outcomes. Randomized trials and observational cohorts of AI-assisted colonoscopy versus non-AI abound but there is a dearth of head-to-head studies evaluating AI system performance. Methods: The aim of our prospective, single-center, pragmatic implementation trial was to perform a 90-day comparative analysis of 2 AI systems: System A (8/20-11/18/2024) and System B (11/19/2024-2/17/2025). We included all screening, surveillance, and positive fecal immunochemical test colonoscopies. The primary outcome was adenoma detection rate (ADR), which included both tubular adenomas and sessile serrated lesions (SSL). Secondary outcomes included the detection rates of tubular adenomas alone, advanced adenomas, SSL, advanced SSL, hyperplastic polyps, and other benign, non-adenomatous lesions, as well as withdrawal time and polyps per colonoscopy (PPC). Results: We evaluated 560 colonoscopies: 357 with System A and 203 with System B. Patient and procedure characteristics were similar between groups. A non-significant trend toward higher ADR was seen with System B (80.3% vs 76.2%, P = 0.26). System A resulted 3.8 PPC versus System B 2.9 (P = 0.005). A notable but non-significant trend in the direction of lower detection rates of benign, non-adenomatous lesions was observed with System B. The systems performed comparably on all other secondary outcomes. Conclusion: System A and System B performed similarly in the real-world setting. ADR for both systems was exceptionally high and while not statistically significant in this population the absolute difference of 4.1% (B > A) may be clinically relevant. System B also resulted in lower rates of benign, non-neoplastic detection rates. This result was not statistically significant, but the lower rate of needless resection is likely clinically meaningful and may also explain the lower PPC for System B. Our study suggests flexibility in choosing a system, leaving room for practical considerations (e.g. user preferences and cost). Our study also invites investigators to look beyond ADR and into the specificity for adenoma detection.