Large language models are less effective at clinical prediction tasks than locally trained machine learning models
These findings suggest that non-fine-tuned LLMs are less effective and robust than locally trained ML for clinical prediction tasks, but they are improving across releases.
Authors
- Chao Yan (ORCID: https://orcid.org/0000-0002-6719-1388)
- Xinmeng Zhang (ORCID: https://orcid.org/0000-0001-7876-0753)
- Yevgeniy Vorobeychik (ORCID: https://orcid.org/0000-0003-2471-5345)
- Benjamin Collins (ORCID: https://orcid.org/0000-0002-6884-3819)
- Zhuohang Li (ORCID: https://orcid.org/0000-0001-5559-4094)
- Murat Kantarcıoğlu (ORCID: https://orcid.org/0000-0001-9795-9063)
- Bradley Malin (ORCID: https://orcid.org/0000-0003-3040-5175)
- Ellen Wright Clayton (ORCID: https://orcid.org/0000-0002-0308-4110)
- You Chen (ORCID: https://orcid.org/0000-0001-8232-8840)
- Katherine Elizabeth Brown (ORCID: https://orcid.org/0000-0003-4443-8541)
Institutions
- Vanderbilt University (US)
- Washington University in St. Louis (US)
- Virginia Tech (US)
- Vanderbilt University Medical Center (US)
Publication Details
- Journal
- Journal of the American Medical Informatics Association
- Published
- 2025-03-08
- DOI
- https://doi.org/10.1093/jamia/ocaf038
- Citations
- 25
- Primary Topic
- Machine Learning in Healthcare
- Type
- article
- Field-Weighted Citation Impact
- 62.02
Funders
- National Science Foundation
- National Institutes of Health