Agentic AI versus pediatric emergency physicians for differential diagnosis of acutely ill and injured children

Background Misdiagnosis and delayed diagnosis of children in pediatric emergency departments lead to complications, and are particularly problematic in settings with limited pediatric expertise. Large language models (LLMs) have shown promise in diagnostic reasoning for adult emergency department patients, but utility for pediatric cases is largely untested. Methods Retrospective study of patients seen in the emergency department of a major urban pediatric academic medical center and subsequently admitted to the operating room or inpatient service. Records from 100 patients were randomly selected, stratified by age. We compared the accuracy of differential diagnoses (up to five, ranked by likelihood) generated by an LLM agent, Dr CaBot (one configuration with literature search, the other without), with those of four attending pediatric emergency medicine physicians (diagnosticians), all masked to outcome. Encounters were presented to diagnosticians stepwise across four touchpoints: triage, the initial physician evaluation (the a priori primary touchpoint), the full emergency department encounter, and the admission. Ground truth diagnosis was determined by two independent, board-certified pediatric emergency physicians (reviewers) based on the entire clinical encounter. The reviewers, masked to source, then assigned each differential a Bond score (0 to 5; 4 or above was high-scoring). Findings The median patient age was 4 years and 41% had a complex chronic condition. Bond score agreement between the two reviewers had a weighted kappa of 0.64. For the primary comparison at the initial physician evaluation touchpoint, the LLM agent and the diagnosticians both achieved high-scoring differentials in 95% of cases (difference 0%, 95%CI: -5.8, 5.8%). No differences were detected at the other touchpoints. Interpretation Dr CaBot and attending pediatric emergency medicine physicians showed similar performance in generating differential diagnoses from clinical documentation, suggesting potential as a diagnostic aid. Whether this performance improves care, particularly outside pediatric specialty centers, requires prospective evaluation.

Authors

Institutions

Publication Details

Journal
medRxiv
Published
2026-11-18
DOI
https://doi.org/10.64898/2026.11.16.26367440
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
preprint

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Agentic AI versus pediatric emergency physicians for differential diagnosis of acutely ill and injured children

Kenneth D. Mandl, Arjun Kumar Manrai, Thomas A. Buckley, Katia C. Genadry et al.
10 links
medRxiv
Artificial Intelligence in Healthcare and Education
preprint

Agentic AI versus pediatric emergency physicians for differential diagnosis of acutely ill and injured children

Kenneth D. Mandl, Arjun Kumar Manrai, Thomas A. Buckley, Katia C. Genadry, Robert G. Hoffmann, Andrew McMurry, Nathan Georgette, Andrew M Fine, Caroline Kahane, Alexandra Geanacopoulos, James Jones
preprint en

Abstract

Background Misdiagnosis and delayed diagnosis of children in pediatric emergency departments lead to complications, and are particularly problematic in settings with limited pediatric expertise. Large language models (LLMs) have shown promise in diagnostic reasoning for adult emergency department patients, but utility for pediatric cases is largely untested. Methods Retrospective study of patients seen in the emergency department of a major urban pediatric academic medical center and subsequently admitted to the operating room or inpatient service. Records from 100 patients were randomly selected, stratified by age. We compared the accuracy of differential diagnoses (up to five, ranked by likelihood) generated by an LLM agent, Dr CaBot (one configuration with literature search, the other without), with those of four attending pediatric emergency medicine physicians (diagnosticians), all masked to outcome. Encounters were presented to diagnosticians stepwise across four touchpoints: triage, the initial physician evaluation (the a priori primary touchpoint), the full emergency department encounter, and the admission. Ground truth diagnosis was determined by two independent, board-certified pediatric emergency physicians (reviewers) based on the entire clinical encounter. The reviewers, masked to source, then assigned each differential a Bond score (0 to 5; 4 or above was high-scoring). Findings The median patient age was 4 years and 41% had a complex chronic condition. Bond score agreement between the two reviewers had a weighted kappa of 0.64. For the primary comparison at the initial physician evaluation touchpoint, the LLM agent and the diagnosticians both achieved high-scoring differentials in 95% of cases (difference 0%, 95%CI: -5.8, 5.8%). No differences were detected at the other touchpoints. Interpretation Dr CaBot and attending pediatric emergency medicine physicians showed similar performance in generating differential diagnoses from clinical documentation, suggesting potential as a diagnostic aid. Whether this performance improves care, particularly outside pediatric specialty centers, requires prospective evaluation.

medRxiv
Boston Children's Hospital (US), Harvard University (US)
National Institutes of Health
Good health and well-being
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon