Assessing the quality of AI-generated clinical notes: validated evaluation of a large language model ambient scribe

LLM-generated Ambient notes demonstrated quality comparable to physician-authored notes across multiple specialties. While Ambient notes were more thorough and better organized, they were also less succinct and more prone to hallucination. The PDQI-9 provides a validated, practical framework for evaluating AI-generated clinical documentation. This quality assessment methodology can inform iterative quality optimization and support the standardization of ambient AI scribes in clinical practice.

Authors

Institutions

Publication Details

Journal
Frontiers in Artificial Intelligence
Published
2025-10-22
DOI
https://doi.org/10.3389/frai.2025.1691499
Citations
7
Primary Topic
Electronic Health Records Systems
Type
article
Field-Weighted Citation Impact
20.56
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Assessing the quality of AI-generated clinical notes: validated evaluation of a large language model ambient scribe

Herprit Mahal, Emil Palm, Astrit Manikantan, Srikanth Subramanya Belwadi et al.
7 citations
Frontiers in Artificial Intelligence
Electronic Health Records Systems
20.56
article

Assessing the quality of AI-generated clinical notes: validated evaluation of a large language model ambient scribe

Herprit Mahal, Emil Palm, Astrit Manikantan, Srikanth Subramanya Belwadi, Mark E. Pepin
article en
7 citations

Abstract

LLM-generated Ambient notes demonstrated quality comparable to physician-authored notes across multiple specialties. While Ambient notes were more thorough and better organized, they were also less succinct and more prone to hallucination. The PDQI-9 provides a validated, practical framework for evaluating AI-generated clinical documentation. This quality assessment methodology can inform iterative quality optimization and support the standardization of ambient AI scribes in clinical practice.

Frontiers in Artificial IntelligenceVol. 8
Maxygen (United States) (US), Cardiovascular Institute of the South (US), Santa Clara Valley Medical Center (US), Colorado Permanente Medical Group (US), Hippocration General Hospital (GR), Stanford University (US)
Openalex Percentile: Top 1%
Electronic Health Records Systems
20.56
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.