Developing and Evaluating Large Language Model–Generated Emergency Medicine Handoff Notes
In this cohort study of 1600 EM patient medical records, LLM-generated EM-to-IP handoff notes were determined superior compared with physician-written summaries via conventional automated evaluation methods, but marginally inferior in usefulness and safety via a novel evaluation framework. This study suggests the importance of a physician-in-loop implementation design for this model and demonstrates an effective strategy to measure preimplementation patient safety of LLM models.
Authors
- Vince Hartman
- Alexander Fortenko (ORCID: https://orcid.org/0000-0002-7502-597X)
- Peter A.D. Steel (ORCID: https://orcid.org/0000-0002-2878-2804)
- Xinyuan Zhang (ORCID: https://orcid.org/0000-0002-4636-1202)
- Matthew McCarty
- Thomas R. Campion (ORCID: https://orcid.org/0000-0001-7624-769X)
- Evan Sholle (ORCID: https://orcid.org/0000-0001-9518-4399)
- Rahul Sharma (ORCID: https://orcid.org/0000-0002-8254-4815)
- Ritika Poddar
Institutions
- Cornell University (US)
- Presbyterian Hospital (US)
- Weill Cornell Medicine (US)
Publication Details
- Journal
- JAMA Network Open
- Published
- 2024-12-03
- DOI
- https://doi.org/10.1001/jamanetworkopen.2024.48723
- Citations
- 47
- Primary Topic
- Emergency and Acute Care Studies
- Type
- article
- Field-Weighted Citation Impact
- 32.47