Quality, safety, and readability of large language model responses to patient-derived questions on tooth whitening, resin infiltration, composite bonding, and porcelain veneers

Patients are increasingly consulting large language model (LLM) platforms for information regarding aesthetic dental procedures prior to clinical appointments. This study evaluated the quality, safety, and readability of responses produced by ChatGPT, Gemini, and Meta AI in relation to patient-derived questions. Forty-two patient-derived questions regarding tooth whitening, resin infiltration, composite bonding, and porcelain veneers were presented to each platform. Two calibrated evaluators independently assessed all 126 responses using a five-domain Primary Quality Rubric, a five-item Modified DISCERN tool, and the Global Quality Score (GQS). The Flesch Reading Ease Score (FRES) and Flesch-Kincaid Grade Level (FKGL) were utilized to evaluate readability and comprehension. Inter-rater reliability was assessed using weighted Cohen's kappa, while between-group differences were evaluated using the Friedman test for repeated measures, followed by paired post-hoc Wilcoxon signed-rank tests and Bonferroni correction. Initial independent inter-rater reliability was slight (pre-consensus kappa varied from 0.011 to 0.191); all subsequent scoring disagreements were successfully resolved through structured consensus discussion. There were no significant differences in accuracy and patient-centeredness among platforms (p = 0.292 and p = 0.071, respectively). Significant differences were identified for Rubric Total (Chi2_F(2) = 20.383, p < 0.001), Modified DISCERN total (Chi2 F(2) = 30.160, p < 0.001), and GQS (Chi2 F(2) = 18.978, p < 0.001). Post-hoc analysis revealed that ChatGPT's scores were significantly lower than both Gemini and Meta AI on Rubric Total, Modified DISCERN, and GQS (all adjusted p < 0.005), but Gemini and Meta AI did not differ significantly except for Actionability, where Meta AI scored higher. The average GQS was 3.60 for ChatGPT, 3.98 for Gemini, and 4.15 for Meta AI. None of the platforms met the recommended Grade 8 readability target, with FKGL scores ranging from 10.2 to 16.5. Meta AI and Gemini outperformed ChatGPT on most quality dimensions relevant to aesthetic dental patient information. No platform consistently met all benchmarks for complete, safe, and accessible responses, underscoring the need for professional oversight when patients consult LLMs for pre-consultation guidance.

Authors

Institutions

Publication Details

Journal
The Saudi Dental Journal
Published
2026-11-16
DOI
https://doi.org/10.1007/s44445-026-00265-6
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Quality, safety, and readability of large language model responses to patient-derived questions on tooth whitening, resin infiltration, composite bonding, and porcelain veneers

Sharma Rahul, Harshita Wadhwa, Vivek Malik, Anmol Jain
4 links
The Saudi Dental Journal
Artificial Intelligence in Healthcare and Education
article

Quality, safety, and readability of large language model responses to patient-derived questions on tooth whitening, resin infiltration, composite bonding, and porcelain veneers

Sharma Rahul, Harshita Wadhwa, Vivek Malik, Anmol Jain
article en

Abstract

Patients are increasingly consulting large language model (LLM) platforms for information regarding aesthetic dental procedures prior to clinical appointments. This study evaluated the quality, safety, and readability of responses produced by ChatGPT, Gemini, and Meta AI in relation to patient-derived questions. Forty-two patient-derived questions regarding tooth whitening, resin infiltration, composite bonding, and porcelain veneers were presented to each platform. Two calibrated evaluators independently assessed all 126 responses using a five-domain Primary Quality Rubric, a five-item Modified DISCERN tool, and the Global Quality Score (GQS). The Flesch Reading Ease Score (FRES) and Flesch-Kincaid Grade Level (FKGL) were utilized to evaluate readability and comprehension. Inter-rater reliability was assessed using weighted Cohen's kappa, while between-group differences were evaluated using the Friedman test for repeated measures, followed by paired post-hoc Wilcoxon signed-rank tests and Bonferroni correction. Initial independent inter-rater reliability was slight (pre-consensus kappa varied from 0.011 to 0.191); all subsequent scoring disagreements were successfully resolved through structured consensus discussion. There were no significant differences in accuracy and patient-centeredness among platforms (p = 0.292 and p = 0.071, respectively). Significant differences were identified for Rubric Total (Chi2_F(2) = 20.383, p < 0.001), Modified DISCERN total (Chi2 F(2) = 30.160, p < 0.001), and GQS (Chi2 F(2) = 18.978, p < 0.001). Post-hoc analysis revealed that ChatGPT's scores were significantly lower than both Gemini and Meta AI on Rubric Total, Modified DISCERN, and GQS (all adjusted p < 0.005), but Gemini and Meta AI did not differ significantly except for Actionability, where Meta AI scored higher. The average GQS was 3.60 for ChatGPT, 3.98 for Gemini, and 4.15 for Meta AI. None of the platforms met the recommended Grade 8 readability target, with FKGL scores ranging from 10.2 to 16.5. Meta AI and Gemini outperformed ChatGPT on most quality dimensions relevant to aesthetic dental patient information. No platform consistently met all benchmarks for complete, safe, and accessible responses, underscoring the need for professional oversight when patients consult LLMs for pre-consultation guidance.

The Saudi Dental JournalVol. 38(12)
Kamineni Institute of Dental Sciences (IN)
Good health and well-being
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon