Clinician Acceptance of Large Language Model–Generated Discharge Summaries

Importance: Hospital discharge summaries are essential for continuity of care, but fragmented electronic health record (EHR) data can compromise their quality and safety. Large language models (LLMs) are increasingly proposed for clinical documentation tasks despite limited evidence regarding their performance in health care settings compared with the current standard of care. Objective: To evaluate clinician acceptability of LLM-generated discharge summaries derived from EHRs compared with routine clinician-written summaries. Design, Setting, and Participants: In this retrospective comparative effectiveness study using a prespecified noninferiority framework, acceptability of LLM-generated discharge summaries was compared with clinician-authored discharge summaries. Admission records from 5 London hospitals for September 1, 2022, to September 30, 2023, were evaluated against the full admission record by practicing physicians. Admissions were eligible only if the clinician-authored discharge summary contained identifiable clinical summary, plan and requested actions, and information given sections. Data analysis was conducted from July 2025 to February 2026. Intervention: LLM-generated summaries were produced using GPT-4o mini (July 18, 2024 release; OpenAI) and a clinician-codesigned prompt. Main Outcomes and Measures: The primary outcome was clinician signoff with no or minor edits, derived from 2 sequential questions: (1) whether the reviewer would sign the summary and (2) whether no, minor, or major edits were required. A 5-percentage-point noninferiority margin was prespecified. Secondary outcomes assessed accuracy, completeness, tone, and clarity (using a 5-point Likert scale: 1, poor; 2, fair; 3, good; 4, very good; and 5, excellent) and potential harmfulness of missing or incorrect information (using a 5-point Likert scale: 1, no harm; 2, minor harm; 3, moderate harm; 4, high harm; and 5, extremely high harm). Results: The study sample included 1294 (649 clinician-authored and 645 LLM-generated) unique case discharge summaries that were evaluated against the full admission record by 12 practicing physicians. Case patients had a median age of 46.0 years (IQR, 26.2-69.8 years), 733 (56.6%) were assigned female at birth, and the median length of stay was 3.0 days (IQR, 1.0-7.0 days). The absolute difference in model-estimated probability of signoff with no or minor edits (LLM generated minus clinician authored) was 22.2 percentage points (95% CI, 19.0-27.5 percentage points); the lower bound exceeded the prespecified noninferiority margin of -5 percentage points, demonstrating noninferiority. LLM-generated summaries received higher ratings for accuracy, completeness, tone, and clarity (adjusted risk differences for scores ≥4 ranged from 19.4 [95% CI, 14.8-24.1] to 29.1 [95% CI, 25.1-33.6] percentage points; all 95% CIs excluded 0). LLM-generated summaries also had a lower proportion of cases rated 3 or greater for potential harm on the clinical summary and plan and requested actions sections (140 of 534 [26.2%] and 124 of 462 [26.8%]) compared with clinician-authored summaries (271 of 601 [45.1%] and 278 of 540 [51.5%]). Conclusions and Relevance: In this study, LLM-generated discharge summaries met the current standard of care, representing a promising early use case for generative artificial intelligence in health care with close clinician oversight.

Authors

Institutions

Publication Details

Journal
JAMA Network Open
Published
2026-11-17
DOI
https://doi.org/10.1001/jamanetworkopen.2026.44496
Citations
1
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
1.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Clinician Acceptance of Large Language Model–Generated Discharge Summaries

Ben Glampson, Benjamin Atkins, Catalina Carenzo, Ina Ko et al.
1 citations
JAMA Network Open
Artificial Intelligence in Healthcare and Education
1.00
article

Clinician Acceptance of Large Language Model–Generated Discharge Summaries

Ben Glampson, Benjamin Atkins, Catalina Carenzo, Ina Ko, Ho Lun Chong, Maite Arribas, Kelsey Flott, Subanki Srisakthivel, Danielle Beleutz, Fidan Damirova, Kathleen Goldsmith, Mazin Mukhaiber, Seham Shaffee, Christian Oldfield, Verity India Prudence Loake, Adna Mohamud, Amna Nadeem, Hutan Momtazian, Ghavin Kuganesan, Maanya Suri, Asmita Raja, Erik Mayer, Tim Orchard
article en
1 citations

Abstract

Importance: Hospital discharge summaries are essential for continuity of care, but fragmented electronic health record (EHR) data can compromise their quality and safety. Large language models (LLMs) are increasingly proposed for clinical documentation tasks despite limited evidence regarding their performance in health care settings compared with the current standard of care. Objective: To evaluate clinician acceptability of LLM-generated discharge summaries derived from EHRs compared with routine clinician-written summaries. Design, Setting, and Participants: In this retrospective comparative effectiveness study using a prespecified noninferiority framework, acceptability of LLM-generated discharge summaries was compared with clinician-authored discharge summaries. Admission records from 5 London hospitals for September 1, 2022, to September 30, 2023, were evaluated against the full admission record by practicing physicians. Admissions were eligible only if the clinician-authored discharge summary contained identifiable clinical summary, plan and requested actions, and information given sections. Data analysis was conducted from July 2025 to February 2026. Intervention: LLM-generated summaries were produced using GPT-4o mini (July 18, 2024 release; OpenAI) and a clinician-codesigned prompt. Main Outcomes and Measures: The primary outcome was clinician signoff with no or minor edits, derived from 2 sequential questions: (1) whether the reviewer would sign the summary and (2) whether no, minor, or major edits were required. A 5-percentage-point noninferiority margin was prespecified. Secondary outcomes assessed accuracy, completeness, tone, and clarity (using a 5-point Likert scale: 1, poor; 2, fair; 3, good; 4, very good; and 5, excellent) and potential harmfulness of missing or incorrect information (using a 5-point Likert scale: 1, no harm; 2, minor harm; 3, moderate harm; 4, high harm; and 5, extremely high harm). Results: The study sample included 1294 (649 clinician-authored and 645 LLM-generated) unique case discharge summaries that were evaluated against the full admission record by 12 practicing physicians. Case patients had a median age of 46.0 years (IQR, 26.2-69.8 years), 733 (56.6%) were assigned female at birth, and the median length of stay was 3.0 days (IQR, 1.0-7.0 days). The absolute difference in model-estimated probability of signoff with no or minor edits (LLM generated minus clinician authored) was 22.2 percentage points (95% CI, 19.0-27.5 percentage points); the lower bound exceeded the prespecified noninferiority margin of -5 percentage points, demonstrating noninferiority. LLM-generated summaries received higher ratings for accuracy, completeness, tone, and clarity (adjusted risk differences for scores ≥4 ranged from 19.4 [95% CI, 14.8-24.1] to 29.1 [95% CI, 25.1-33.6] percentage points; all 95% CIs excluded 0). LLM-generated summaries also had a lower proportion of cases rated 3 or greater for potential harm on the clinical summary and plan and requested actions sections (140 of 534 [26.2%] and 124 of 462 [26.8%]) compared with clinician-authored summaries (271 of 601 [45.1%] and 278 of 540 [51.5%]). Conclusions and Relevance: In this study, LLM-generated discharge summaries met the current standard of care, representing a promising early use case for generative artificial intelligence in health care with close clinician oversight.

JAMA Network OpenVol. 9(11)
Imperial College Healthcare NHS Trust (GB), National Institute for Health and Care Research (GB), NIHR Imperial Biomedical Research Centre (GB), Imperial College London (GB)
National Institute for Health and Care Research, NIHR Imperial Biomedical Research Centre
Good health and well-being
Openalex Percentile: Top 26%
Artificial Intelligence in Healthcare and Education
1.00
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon