Large language model diagnostic performance across informant sources in child and adolescent psychiatry: a prospective multicenter study

Large language models (LLMs) are increasingly considered for mental health assessment, yet child and adolescent psychiatric diagnosis depends on information from multiple informants. It remains unclear how the source and richness of clinical information influence LLM diagnostic performance. This study evaluated diagnostic accuracy across child, parent, combined, and clinician-enriched information conditions and compared performance across three widely used LLMs. This prospective multicenter study included 220 children and adolescents aged 1–17 years presenting to two university child and adolescent psychiatry clinics in Turkey. Presenting complaints were obtained separately from children and parents. Four input conditions were created: child only, parent only, child plus parent, and child plus parent supplemented with physician-derived clinical information. Three LLMs were evaluated using independent single-turn API queries, followed by two repeat runs to assess reproducibility. Clinician-established DSM-5 diagnoses served as the reference standard. Performance was assessed using Top-1 and Top-3 accuracy, mean reciprocal rank, Cohen’s kappa, F1 scores, and sensitivity and specificity for detecting any psychiatric diagnosis. Models and conditions were compared with paired tests and generalized estimating equation models. Top-1 accuracy was lowest in the child-only condition (35.5–44.1% across models), increased to 65.9–69.1% with parent information, and was highest when physician-derived information was added (78.6–83.2%). Adding the child narrative to the parent narrative did not significantly improve accuracy (69.1–71.8%). Full-information Top-3 accuracy ranged from 88.2 to 91.8%. Model differences were significant only in the child-only condition. With parent narratives, sensitivity for any psychiatric diagnosis exceeded 93%, but specificity was 14.3–42.9%; physician-derived information increased specificity to 52.4–61.9%. Age was associated with higher accuracy only in the child-only condition (OR = 1.164 per year, 95% CI 1.071–1.264). Two repeat runs reproduced the main information-condition findings; between-model differences were less consistent. LLM diagnostic performance varied with the available clinical information, with the highest accuracy observed when physician-derived findings were included. Adding child narratives to parent reports did not significantly improve accuracy, and the models frequently assigned diagnoses to children without a psychiatric disorder. These observational findings are based on routine clinician-established diagnoses and do not demonstrate improvements in clinical decision-making or patient outcomes.

Authors

Institutions

Publication Details

Journal
Scientific Reports
Published
2026-11-20
DOI
https://doi.org/10.1038/s41598-026-81830-1
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Large language model diagnostic performance across informant sources in child and adolescent psychiatry: a prospective multicenter study

Hakan Kayış, Akın Tahıllıoğlu, Ayşegül Tuğba Hıra Selen, Çınar Gedizlioğlu
8 links
Scientific Reports
Artificial Intelligence in Healthcare and Education
article

Large language model diagnostic performance across informant sources in child and adolescent psychiatry: a prospective multicenter study

Hakan Kayış, Akın Tahıllıoğlu, Ayşegül Tuğba Hıra Selen, Çınar Gedizlioğlu
article en

Abstract

Large language models (LLMs) are increasingly considered for mental health assessment, yet child and adolescent psychiatric diagnosis depends on information from multiple informants. It remains unclear how the source and richness of clinical information influence LLM diagnostic performance. This study evaluated diagnostic accuracy across child, parent, combined, and clinician-enriched information conditions and compared performance across three widely used LLMs. This prospective multicenter study included 220 children and adolescents aged 1–17 years presenting to two university child and adolescent psychiatry clinics in Turkey. Presenting complaints were obtained separately from children and parents. Four input conditions were created: child only, parent only, child plus parent, and child plus parent supplemented with physician-derived clinical information. Three LLMs were evaluated using independent single-turn API queries, followed by two repeat runs to assess reproducibility. Clinician-established DSM-5 diagnoses served as the reference standard. Performance was assessed using Top-1 and Top-3 accuracy, mean reciprocal rank, Cohen’s kappa, F1 scores, and sensitivity and specificity for detecting any psychiatric diagnosis. Models and conditions were compared with paired tests and generalized estimating equation models. Top-1 accuracy was lowest in the child-only condition (35.5–44.1% across models), increased to 65.9–69.1% with parent information, and was highest when physician-derived information was added (78.6–83.2%). Adding the child narrative to the parent narrative did not significantly improve accuracy (69.1–71.8%). Full-information Top-3 accuracy ranged from 88.2 to 91.8%. Model differences were significant only in the child-only condition. With parent narratives, sensitivity for any psychiatric diagnosis exceeded 93%, but specificity was 14.3–42.9%; physician-derived information increased specificity to 52.4–61.9%. Age was associated with higher accuracy only in the child-only condition (OR = 1.164 per year, 95% CI 1.071–1.264). Two repeat runs reproduced the main information-condition findings; between-model differences were less consistent. LLM diagnostic performance varied with the available clinical information, with the highest accuracy observed when physician-derived findings were included. Adding child narratives to parent reports did not significantly improve accuracy, and the models frequently assigned diagnoses to children without a psychiatric disorder. These observational findings are based on routine clinician-established diagnoses and do not demonstrate improvements in clinical decision-making or patient outcomes.

Scientific Reports
İzmir University of Economics (TR), Zonguldak Bülent Ecevit University (TR), Bakırçay Üniversitesi (TR), Konya City Hospital (TR)
Good health and well-being
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon