Assessment of ChatGPT responses to nutrition queries of patients with inflammatory bowel disease

Considering the critical role of diet in the management of inflammatory bowel disease (IBD), patients with IBD often seek nutrition-related information from various sources. With the increasing application of artificial intelligence (AI) chatbots across multiple fields, their use as a source of patient educational material has also expanded. However, the readability, accuracy, and comprehensibility of data provided by AI chatbots have not yet been fully established. Therefore, this study was conducted to evaluate the readability, scientific accuracy, and comprehensibility of AI-generated responses to the nutritional queries of patients with IBD. Twenty frequently asked questions about nutrition and IBD were retrieved from Google and were posed to Chat Generative Pretrained Transformer (ChatGPT), version 3.5. The readability of ChatGPT-generated responses was assessed using three validated measures: Flesch Reading Ease Score (FRES), the Flesch Kincaid Grade Level (FKGL), and the Simple Measure of Gobbledygook (SMOG). Scientific accuracy and comprehensibility were evaluated by nine nutritionists and dietitians specializing in IBD. Regarding readability, ChatGPT-generated responses were difficult to read, with FRES = 44.54, FKGL = 11.6, and SMOG = 13.8. Additionally, ChatGPT-generated responses to patient questions were generally scientifically accurate (mean score = 4.2 ± 0.5) and comprehensible (mean score 4.3 ± 0.6). However, both scientific accuracy and comprehensibility decreased when questions became more technical and complex. Overall, the assessment of AI-generated responses demonstrated low readability. However, in terms of scientific accuracy and comprehensibility, the AI chatbot provided relatively accurate and understandable information regarding the nutritional information needs of patients with IBD. Further studies are needed to confirm these findings.