Reliability and readability of AI platforms for pediatric health advice: a comparative analysis

Large language models (LLMs) are increasingly used as a first-line source of information for everyday questions, including pediatric health guidance, yet the quality and readability of their outputs remain uncertain. This study aimed to comparatively evaluate the clinical reliability, quality, and readability of responses generated by four widely used LLMs to real-world, caregiver-style pediatric prompts. A curated set of 28 caregiver-style pediatric questions spanning six common themes was posed on August 5, 2025, to four widely used LLMs (ChatGPT-4o, Gemini 2.5 Pro, Grok-4, DeepSeek-V2), with one fresh session per question-model pair; the first, unedited outputs were retained. Under blinded conditions, responses were scored by four pediatricians using a modified DISCERN instrument (reliability/structure) and a global quality score (perceived usefulness). Readability was assessed with standard indices and length measures. Quality varied across platforms. Grok achieved the highest DISCERN scores, indicating stronger reliability and structural rigor, whereas Gemini received the highest global quality ratings. DeepSeek was consistently rated lower by experts but yielded the most readable outputs; ChatGPT showed intermediate performance. None of the models consistently met health-literacy targets for patient materials (approximately 6th-8th grade reading level). Grok generated the longest and most complex responses (often college level), while DeepSeek and Gemini produced comparatively simpler, more concise text. Across platforms, most responses were classified as moderate in reliability and usefulness; overtly unsafe advice was not identified. These findings suggest that current LLMs provide moderately useful pediatric health information but require improvements in readability, sourcing, and consistency before routine patient-facing use.