Quality, Reliability, and Readability of Artificial Intelligence-Generated Information on Peri-Implant Diseases: A Comparative Study

Background/Objectives: The increasing use of dental implants has been accompanied by a rise in peri-implant diseases. As artificial intelligence (AI)-based chatbots are increasingly used to obtain health information, this study evaluated the reliability, quality, usefulness, and readability of responses generated by ChatGPT (GPT-4), Copilot (Smart GPT-5), and Gemini (3.0 Flash) to questions about peri-implant diseases. Methods: Thirty-two AI-generated patient-oriented questions about peri-implant diseases were submitted to each chatbot. The responses were evaluated using the Global Quality Score (GQS), a 4-point usefulness scale, the modified DISCERN (mDISCERN) tool, the Flesch Reading Ease (FRE), and the Flesch-Kincaid Grade Level (FKGL). Inter-rater agreement before consensus was assessed using single-measure, absolute-agreement intraclass correlation coefficients (ICCs). Statistical analyses included the Shapiro-Wilk, Kruskal-Wallis, post hoc Bonferroni, and Spearman correlation tests. Results: ChatGPT and Gemini had higher FRE scores and lower FKGL scores than Copilot (all adjusted p < 0.001). Copilot and Gemini had higher mDISCERN scores than ChatGPT (both adjusted p < 0.001), with no significant difference between Copilot and Gemini (adjusted p = 0.129). GQS and usefulness scores did not differ significantly across chatbots. Across the three chatbots, Spearman correlation coefficients ranged from −0.973 to −0.844 for the correlation between FRE and FKGL scores and from −0.773 to −0.509 for the correlation between GQS and usefulness scores. Conclusions: ChatGPT and Gemini produced easier-to-read responses than Copilot, while Copilot and Gemini scored higher than ChatGPT on mDISCERN. The chatbots showed no significant differences in overall quality or usefulness scores. Given their potential limitations in clinical depth, AI-generated responses should be used only as supplementary educational resources and should not replace professional medical advice.

Authors

Institutions

Publication Details

Journal
Diagnostics
Published
2026-11-19
DOI
https://doi.org/10.3390/diagnostics16223887
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Quality, Reliability, and Readability of Artificial Intelligence-Generated Information on Peri-Implant Diseases: A Comparative Study

Selin Gaş, Tuğçe Paksoy, Bilge Meraci, Sevda Altınay Uncu
Diagnostics
Artificial Intelligence in Healthcare and Education
article

Quality, Reliability, and Readability of Artificial Intelligence-Generated Information on Peri-Implant Diseases: A Comparative Study

Selin Gaş, Tuğçe Paksoy, Bilge Meraci, Sevda Altınay Uncu
article en

Abstract

Background/Objectives: The increasing use of dental implants has been accompanied by a rise in peri-implant diseases. As artificial intelligence (AI)-based chatbots are increasingly used to obtain health information, this study evaluated the reliability, quality, usefulness, and readability of responses generated by ChatGPT (GPT-4), Copilot (Smart GPT-5), and Gemini (3.0 Flash) to questions about peri-implant diseases. Methods: Thirty-two AI-generated patient-oriented questions about peri-implant diseases were submitted to each chatbot. The responses were evaluated using the Global Quality Score (GQS), a 4-point usefulness scale, the modified DISCERN (mDISCERN) tool, the Flesch Reading Ease (FRE), and the Flesch-Kincaid Grade Level (FKGL). Inter-rater agreement before consensus was assessed using single-measure, absolute-agreement intraclass correlation coefficients (ICCs). Statistical analyses included the Shapiro-Wilk, Kruskal-Wallis, post hoc Bonferroni, and Spearman correlation tests. Results: ChatGPT and Gemini had higher FRE scores and lower FKGL scores than Copilot (all adjusted p < 0.001). Copilot and Gemini had higher mDISCERN scores than ChatGPT (both adjusted p < 0.001), with no significant difference between Copilot and Gemini (adjusted p = 0.129). GQS and usefulness scores did not differ significantly across chatbots. Across the three chatbots, Spearman correlation coefficients ranged from −0.973 to −0.844 for the correlation between FRE and FKGL scores and from −0.773 to −0.509 for the correlation between GQS and usefulness scores. Conclusions: ChatGPT and Gemini produced easier-to-read responses than Copilot, while Copilot and Gemini scored higher than ChatGPT on mDISCERN. The chatbots showed no significant differences in overall quality or usefulness scores. Given their potential limitations in clinical depth, AI-generated responses should be used only as supplementary educational resources and should not replace professional medical advice.

DiagnosticsVol. 16(22)
İstanbul Gelişim Üniversitesi (TR), Sağlık Bilimleri Üniversitesi (TR), Bolu Abant İzzet Baysal University (TR)
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon