Evaluating readability of AI-generated pediatric dental education materials

Background General-purpose large language models (LLMs) have become a popular source of oral health information. However, the patient education materials (PEMs) generated by these models often pose readability challenges. This study aimed to: (1) compare the readability of PEMs generated by six AI chatbots in response to ten frequently asked questions on pediatric oral health; and (2) determine whether the readability of these texts meets the level recommended by the American Medical Association (AMA). Methods Ten sample questions on pediatric oral health were compiled. Each question was submitted once to each of six AI chatbots (GPT-5 Mini, Gemini 3 Flash, Claude Haiku 4.5, DeepSeek-V3, Kimi K2.5, and Qwen3-Max), generating a corpus of 60 texts in a blocked (repeated-measures) design. Six readability indices — Flesch Reading Ease (FRE), Flesch–Kincaid Grade Level (FKGL), Gunning Fog Index (GFI), Coleman–Liau Index (CLI), Simple Measure of Gobbledygook (SMOG), and Automated Readability Index (ARI) — were calculated for every text. Each index was analysed separately using the Friedman test, with the ten questions as blocks and the six models as conditions, followed by Dunn's multiple comparison test. Kendall's coefficient of concordance (W) was calculated to quantify the consistency of model ranking across questions. Results DeepSeek-V3 showed the highest readability (easiest texts) and Gemini 3 Flash the second highest, whereas Claude Haiku 4.5 showed the lowest readability across all six indices; GPT-5 Mini, Qwen3-Max, and Kimi K2.5 occupied intermediate positions. All six Friedman tests were statistically significant ( χ 2 (5) = 31.77–35.94, all p < 0.0001), and Kendall's W ranged from 0.635 to 0.719, indicating that the relative ranking of the six models was broadly consistent across the ten questions. Against the AMA-recommended sixth-grade reading level, all models exceeded the recommended difficulty on every index; even against a more lenient seventh-grade threshold, only the ARI median of DeepSeek-V3 (6.5) fell below 7. Conclusions The PEMs generated by the six LLMs were written at a reading level above the AMA recommendation, and reading difficulty differed systematically among models. Readability should be explicitly controlled through prompt design, model optimisation, and professional review before such materials are used in pediatric oral health education.