Readability and Quality Analysis of ChatGPT o1’s Responses on Colorectal Cancer: A Study of an AI Language Model

Aim: This study aimed to evaluate the readability and quality of the responses provided by the ChatGPT model to the most frequently searched questions by patients about colorectal cancer on the internet.Methods: The 20 most frequently searched topics related to colorectal cancer were identified from Google and Yandex search engine statistics.These topics were posed to the ChatGPT o1 model, and the obtained responses were analyzed for readability using the Ateman and etinkaya-Uzun readability formulas.Quality assessment was performed using the DISCERN instrument and the Global Quality Score (GQS).Statistical analyses included Pearson correlation and one-way ANOVA tests. Results:The average word count of the responses was 654.9 [standard deviation (SD)=221.62].According to the Ateman readability score, the mean score was 55.45 (SD=6.06,medium difficulty readability), and according to the etinkaya-Uzun score, it was 85.53 (SD=4.0, 5 th -7 th grade level, independently readable).The mean total DISCERN score was 54.55 (SD=5.75, which indicates good quality), and the mean GQS was 4.35 (SD=0.75, which suggests between good and excellent).No significant correlation was found between DISCERN and GQS scores (p=0.831). Conclusion:The responses provided by the ChatGPT o1 model to patients' most frequently asked questions about colorectal cancer have mediumlevel readability and good-quality content.Therefore, it can be considered a helpful resource for patients seeking information.