Large language models for error detection in radiology reports: a comparative analysis between closed-source and privacy-compliant open-source models
Question Can privacy-compliant open-source large language models (LLMs) match the error-detection performance of commercial non-privacy-compliant closed-source models in radiology reports? Findings Closed-source LLMs achieved slightly higher accuracy in detecting radiology report errors than open-source models, with Llama 3-70b yielding the best results among the open-source models. Clinical relevance Open-source LLMs offer a privacy-compliant alternative for automated error detection in radiology reports, improving clinical workflow efficiency while ensuring patient data confidentiality. Further refinement could enhance their accuracy, contributing to better diagnosis and patient care.
Authors
- Maike Theis (ORCID: https://orcid.org/0000-0003-2666-9928)
- Daniel Kuetting (ORCID: https://orcid.org/0000-0001-6520-1020)
- Julian Alexander Luetkens (ORCID: https://orcid.org/0000-0002-7715-4636)
- Dmitrij Kravchenko (ORCID: https://orcid.org/0000-0003-0331-2045)
- Tatjana Dell (ORCID: https://orcid.org/0000-0002-2835-8560)
- Claus Christian Pieper (ORCID: https://orcid.org/0000-0002-0893-1200)
- Christoph Endler (ORCID: https://orcid.org/0000-0002-5571-5527)
- Sebastian Nowak (ORCID: https://orcid.org/0000-0001-5175-5559)
- Narine Mesropyan (ORCID: https://orcid.org/0000-0002-5871-3965)
- Alexander Isaak (ORCID: https://orcid.org/0000-0003-2016-6166)
- Alois M. Sprinkart (ORCID: https://orcid.org/0000-0002-1435-9562)
- Babak Salam
- Claire Stüwe
Institutions
- University Hospital Bonn (DE)
Publication Details
- Journal
- European Radiology
- Published
- 2025-02-20
- DOI
- https://doi.org/10.1007/s00330-025-11438-y
- Citations
- 22
- Primary Topic
- Radiology practices and education
- Type
- article
- Field-Weighted Citation Impact
- 29.92
Funders
- Ernst und Berta Grimmke Stiftung