Quality, readability, and potential safety risks of epididymitis information generated by six artificial intelligence chatbots: a cross-sectional comparative study

Background AI chatbots are increasingly used for medical information, but the quality, readability, and safety of disease-specific responses remain uncertain. We compared epididymitis information generated by six widely accessible AI chatbots. Methods Nine of 25 Google Trends queries identified worldwide from August 2021 to August 2026 were each submitted once to ChatGPT, Gemini, Perplexity, Copilot, Kimi, and DeepSeek. Responses were assessed with DISCERN, EQIP, JAMA, GQS, and six readability indices. Between-model differences in DISCERN, EQIP, and the readability indices were assessed using Friedman tests; significant outcomes underwent paired Wilcoxon signed-rank tests, with Holm correction for multiple comparisons. JAMA and GQS were summarized descriptively. Potentially harmful misinformation was assessed exploratorily. Results DISCERN, EQIP, and all six readability indices showed significant overall model effects after Holm correction, but no post hoc pairwise comparison remained statistically significant. Copilot had the highest median EQIP score (70.00). JAMA scores were 0 in 52/54 responses (96.3%), and none of the 54 responses met sixth-grade readability benchmarks. Potentially harmful information occurred in 4/54 responses (7.4%), all involving antimicrobial recommendations. Conclusion In this exploratory cross-sectional snapshot, the chatbots differed in selected quality measures but shared poor source transparency and excessive reading complexity. They may supplement general patient information but should not replace professional assessment or guideline-based care.

Authors

Institutions

Publication Details

Journal
Frontiers in Medicine
Published
2026-11-13
DOI
https://doi.org/10.3389/fmed.2026.1991323
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Quality, readability, and potential safety risks of epididymitis information generated by six artificial intelligence chatbots: a cross-sectional comparative study

Baokun Fan, Cheng Zhou
6 links
Frontiers in Medicine
Artificial Intelligence in Healthcare and Education
article

Quality, readability, and potential safety risks of epididymitis information generated by six artificial intelligence chatbots: a cross-sectional comparative study

Baokun Fan, Cheng Zhou
article en

Abstract

Background AI chatbots are increasingly used for medical information, but the quality, readability, and safety of disease-specific responses remain uncertain. We compared epididymitis information generated by six widely accessible AI chatbots. Methods Nine of 25 Google Trends queries identified worldwide from August 2021 to August 2026 were each submitted once to ChatGPT, Gemini, Perplexity, Copilot, Kimi, and DeepSeek. Responses were assessed with DISCERN, EQIP, JAMA, GQS, and six readability indices. Between-model differences in DISCERN, EQIP, and the readability indices were assessed using Friedman tests; significant outcomes underwent paired Wilcoxon signed-rank tests, with Holm correction for multiple comparisons. JAMA and GQS were summarized descriptively. Potentially harmful misinformation was assessed exploratorily. Results DISCERN, EQIP, and all six readability indices showed significant overall model effects after Holm correction, but no post hoc pairwise comparison remained statistically significant. Copilot had the highest median EQIP score (70.00). JAMA scores were 0 in 52/54 responses (96.3%), and none of the 54 responses met sixth-grade readability benchmarks. Potentially harmful information occurred in 4/54 responses (7.4%), all involving antimicrobial recommendations. Conclusion In this exploratory cross-sectional snapshot, the chatbots differed in selected quality measures but shared poor source transparency and excessive reading complexity. They may supplement general patient information but should not replace professional assessment or guideline-based care.

Frontiers in MedicineVol. 13
Ningbo University (CN)
Good health and well-being
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon