Assessing the Limitations of Large Language Models in Clinical Practice Guideline–Concordant Treatment Decision-Making on Real-World Data: Retrospective Study
Even advanced LLMs require extensively curated input for informed treatment decisions. Unreliable responses, bias, and hallucinations pose significant health risks and highlight the need for caution in applying LLMs to real-world clinical decision-making.
Authors
- Tobias Daniel Trippel (ORCID: https://orcid.org/0000-0001-8575-0019)
- Tobias Roeschl (ORCID: https://orcid.org/0000-0002-8376-3366)
- Marie Hoffmann (ORCID: https://orcid.org/0000-0001-9807-8292)
- Jörg Kempfert (ORCID: https://orcid.org/0000-0002-2009-6375)
- Matthias I. Gröschel (ORCID: https://orcid.org/0000-0002-2509-3034)
- Volkmar Falk (ORCID: https://orcid.org/0000-0002-7911-8620)
- Gerhard Hindricks (ORCID: https://orcid.org/0000-0002-0132-0773)
- Henryk Dreger (ORCID: https://orcid.org/0000-0003-1909-4329)
- Axel Unbehaun (ORCID: https://orcid.org/0000-0003-2851-695X)
- Benjamin O’Brien (ORCID: https://orcid.org/0000-0001-7024-2337)
- Felix Balzer (ORCID: https://orcid.org/0000-0003-1575-2056)
- Nils Hinrichs (ORCID: https://orcid.org/0000-0002-1989-6156)
- Djawid Hashemi (ORCID: https://orcid.org/0000-0003-3933-7092)
- Alexander Meyer (ORCID: https://orcid.org/0000-0002-6944-2478)
- Christoph Klein (ORCID: https://orcid.org/0009-0003-1016-7971)
- Felix Rarreck (ORCID: https://orcid.org/0000-0001-5337-977X)
Institutions
- St Bartholomew's Hospital (GB)
- Humboldt-Universität zu Berlin (DE)
- Deutsches Herzzentrum der Charité (DE)
- German Centre for Cardiovascular Research (DE)
- Berlin Institute of Health at Charité - Universitätsmedizin Berlin (DE)
- Technische Universität Berlin (DE)
- École Polytechnique Fédérale de Lausanne (CH)
- Freie Universität Berlin (DE)
- Charité - Universitätsmedizin Berlin (DE)
Publication Details
- Journal
- JMIRx Med
- Published
- 2025-11-03
- DOI
- https://doi.org/10.2196/74899
- Citations
- 1
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.65