Comparative performance and safety of three large language models in pediatric paracetamol poisoning management

Abstract Background Paracetamol overdose is a common reason for pediatric poisoning consultations and emergency department visits, and. Large language models (LLMs) are increasingly being explored for clinical decision support, but evidence on pediatric poisoning is scarce. We compared the performance and safety of three contemporary LLMs for managing pediatric paracetamol poisoning using guideline-based simulated scenarios. Methods In this cross-sectional, comparative, scenario-based study, 30 hypothetical pediatric paracetamol poisoning scenarios were presented to ChatGPT 5.5, Claude Sonnet 4.6, and Gemini 3 Flash via their public browser-based interfaces, with a standardized single-turn prompt (90 responses). Blinded to model identity, a three-clinician expert panel scored each response by consensus using a study-specific Artificial Intelligence Evaluation Score for Clinical Poisoning Vignettes (AIES-CPV; range 5–25), an eight-item key-action checklist (0–8), the Global Quality Scale (GQS; 1–5), reference-standard concordance, and the presence of critical safety errors, against a predefined reference algorithm derived primarily from the Australia and New Zealand guidelines. Models were compared using the Friedman test (with Wilcoxon signed-rank tests and a Bonferroni correction for pairwise comparisons) and the Cochran Q test for critical error rates. Results than ChatGPT 5.5 and Gemini 3 Flash (Friedman p < 0.001 for all); ChatGPT 5.5 and Gemini 3 Flash did not differ significantly. Claude Sonnet 4.6 produced no critical safety errors and was concordant with the reference standard in all responses, whereas one critical error occurred with ChatGPT 5.5 (1/30) and three with Gemini 3 Flash (3/30); the difference in critical-error rates was not statistically significant (Cochran Q = 4.67, p = 0.097). Conclusions Among three LLMs evaluated on pediatric paracetamol poisoning scenarios, Claude Sonnet 4.6 showed the highest guideline-based accuracy, key-action performance, and overall quality, with no critical safety errors. Contemporary LLMs may serve as adjunctive decision-support tools in pediatric toxicology, but given inter-model safety differences and the risk of confident errors in high-stakes decisions, they should be used only under expert supervision and with clinical verification.