Currently Available Large Language Models Do Not Provide Musculoskeletal Treatment Recommendations That Are Concordant With Evidence‐Based Clinical Practice Guidelines
Although leading LLMs generally provide recommendations concordant with CPGs, a substantial error rate exists, and the proportion of recommendations that do not align with these CPGs suggests that LLMs are not trustworthy clinical support tools at this time. Each off-the-shelf, closed-source LLM has strengths and weaknesses. Future research should evaluate and compare multiple LLMs to avoid bias associated with narrow evaluation of few models as observed in the current literature.
Authors
- Benedict U. Nwachukwu (ORCID: https://orcid.org/0000-0002-6170-7769)
- Kyle N. Kunze (ORCID: https://orcid.org/0000-0002-0363-3482)
- Answorth A. Allen (ORCID: https://orcid.org/0000-0003-0346-4735)
- David W. Altchek (ORCID: https://orcid.org/0000-0002-7964-3513)
- Nathan H. Varady (ORCID: https://orcid.org/0000-0001-8943-0987)
- Joshua S. Dines (ORCID: https://orcid.org/0000-0002-2135-9317)
- Riley J. Williams (ORCID: https://orcid.org/0000-0003-2991-7173)
Institutions
- Hospital for Special Surgery (US)
Publication Details
- Journal
- Arthroscopy The Journal of Arthroscopic and Related Surgery
- Published
- 2024-08-20
- DOI
- https://doi.org/10.1016/j.arthro.2024.07.040
- Citations
- 33
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 3.97