Evaluating Retrieval Augmented Generation-enhanced Large Language Models for Question Answering On German Neurovascular Guidelines
RAG significantly improves LLM accuracy for medical guideline question answering compared to the inherent knowledge of pretrained LLMs alone while still showing significant error rates. Improved accuracy and confidence metrics are needed for safer implementation in clinical routine. Additionally, our results demonstrate the strong performance of general LLMs in medical question answering for non-English languages, such as German, even without specific training.
Authors
- Marius Vach (ORCID: https://orcid.org/0000-0001-7161-4462)
- Christian Rubbert (ORCID: https://orcid.org/0000-0002-9461-1173)
- Michael Gliem (ORCID: https://orcid.org/0000-0002-0446-9776)
- Vivien Lorena Ivan (ORCID: https://orcid.org/0000-0002-0173-209X)
- Frederik Hauke (ORCID: https://orcid.org/0000-0003-3434-5720)
- Julian Caspers (ORCID: https://orcid.org/0000-0002-1354-3667)
- Daniel J. Weiss
- Christian Boschenriedter
Institutions
- Düsseldorf University Hospital (DE)
- Heinrich Heine University Düsseldorf (DE)
Publication Details
- Journal
- Clinical Neuroradiology
- Published
- 2025-09-02
- DOI
- https://doi.org/10.1007/s00062-025-01562-z
- Citations
- 2
- Primary Topic
- Topic Modeling
- Type
- article
- Field-Weighted Citation Impact
- 4.96
Funders
- Deutsche Forschungsgemeinschaft
- Medizinische Fakultät, Heinrich-Heine-Universität Düsseldorf