Medical large language models are susceptible to targeted misinformation attacks
Large language models (LLMs) have broad medical knowledge and can reason about medical information across many domains, holding promising potential for diverse medical applications in the near future. In this study, we demonstrate a concerning vulnerability of LLMs in medicine. Through targeted manipulation of just 1.1% of the weights of the LLM, we can deliberately inject incorrect biomedical facts. The erroneous information is then propagated in the model's output while maintaining performance on other biomedical tasks. We validate our findings in a set of 1025 incorrect biomedical facts. This peculiar susceptibility raises serious security and trustworthiness concerns for the application of LLMs in healthcare settings. It accentuates the need for robust protective measures, thorough verification mechanisms, and stringent management of access to these models, ensuring their reliable and safe use in medical practice.
Authors
- Christoph Haarburger (ORCID: https://orcid.org/0000-0002-2405-640X)
- Keno K. Bressem (ORCID: https://orcid.org/0000-0001-9249-8624)
- Daniel Truhn (ORCID: https://orcid.org/0000-0002-9605-0728)
- Jens Kleesiek (ORCID: https://orcid.org/0000-0001-8686-0682)
- Tianyu Han (ORCID: https://orcid.org/0000-0002-8636-6462)
- Gustav Müller‐Franzes (ORCID: https://orcid.org/0000-0002-7413-2570)
- Firas Khader (ORCID: https://orcid.org/0000-0001-5089-3589)
- Christiane Katharina Kuhl (ORCID: https://orcid.org/0000-0001-8696-2363)
- Sebastian Foersch (ORCID: https://orcid.org/0000-0002-4740-6900)
- Jakob Nikolas Kather (ORCID: https://orcid.org/0000-0002-3730-5348)
- Sven Nebelung (ORCID: https://orcid.org/0000-0002-5267-9962)
- Tianci Wang (ORCID: https://orcid.org/0000-0002-6203-2310)
Institutions
- Johannes Gutenberg University Mainz (DE)
- Heidelberg University (DE)
- University Hospital Heidelberg (DE)
- Humboldt-Universität zu Berlin (DE)
- University Medical Center of the Johannes Gutenberg University Mainz (DE)
- Fresenius (Germany) (DE)
- National Center for Tumor Diseases (DE)
- Universitätsklinikum Aachen (DE)
- Berlin Institute of Health at Charité - Universitätsmedizin Berlin (DE)
- Freie Universität Berlin (DE)
- Charité - Universitätsmedizin Berlin (DE)
- RWTH Aachen University (DE)
Publication Details
- Journal
- npj Digital Medicine
- Published
- 2024-10-23
- DOI
- https://doi.org/10.1038/s41746-024-01282-7
- Citations
- 40
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 4.81
Funders
- National Institute for Health and Care Research
- European Commission
- Deutscher Akademischer Austauschdienst
- Bundesministerium für Bildung und Forschung
- Bundesministerium für Gesundheit
- Deutsche Krebshilfe
- Gemeinsame Bundesausschuss
- National Institutes of Health
- HORIZON EUROPE Framework Programme
- Leeds Biomedical Research Centre