Benchmarking public large language model responses to patient-facing inflammatory bowel disease questions: informational quality, transparency proxies, and readability

Background: Patient-facing large language model (LLM) outputs for inflammatory bowel disease (IBD) must be decision-relevant, readable, and verifiable. Methods: official web interfaces under default settings (privacy mode; new chat per prompt). Two blinded raters evaluated informational quality and completeness (using DISCERN, EQIP, and the Global Quality Scale), transparency proxies (based on JAMA benchmark criteria), and readability through the Automated Readability Index, Flesch Reading Ease, Gunning Fog Index, Flesch-Kincaid Grade Level, Coleman-Liau Index, and SMOG. Overall differences were assessed using within-question paired Friedman tests with Holm adjustment, and effect size was quantified with Kendall's W. Results: = 0.238-0.702). Conclusion: Under default settings, publicly available LLMs exhibit variable informational quality for IBD but consistently poor transparency and readability. Patient-facing deployment should mandate provenance, currency, and disclosure fields, as well as outputs targeted to appropriate grade levels.