Prompt injection attacks on vision language models in oncology
Vision-language artificial intelligence models (VLMs) possess medical knowledge and can be employed in healthcare in numerous ways, including as image interpreters, virtual scribes, and general decision support systems. However, here, we demonstrate that current VLMs applied to medical tasks exhibit a fundamental security flaw: they can be compromised by prompt injection attacks. These can be used to output harmful information just by interacting with the VLM, without any access to its parameters. We perform a quantitative study to evaluate the vulnerabilities to these attacks in four state of the art VLMs: Claude-3 Opus, Claude-3.5 Sonnet, Reka Core, and GPT-4o. Using a set of N = 594 attacks, we show that all of these models are susceptible. Specifically, we show that embedding sub-visual prompts in manifold medical imaging data can cause the model to provide harmful output, and that these prompts are non-obvious to human observers. Thus, our study demonstrates a key vulnerability in medical VLMs which should be mitigated before widespread clinical adoption.
Authors
- Daniel Truhn (ORCID: https://orcid.org/0000-0002-9605-0728)
- Isabella C. Wiest (ORCID: https://orcid.org/0009-0004-4848-614X)
- Jan Clusmann (ORCID: https://orcid.org/0000-0003-2925-8438)
- Carolin Victoria Schneider (ORCID: https://orcid.org/0000-0002-6728-9246)
- Dyke Ferber (ORCID: https://orcid.org/0009-0006-6195-9276)
- Sebastian Foersch (ORCID: https://orcid.org/0000-0002-4740-6900)
- Jakob Nikolas Kather (ORCID: https://orcid.org/0000-0002-3730-5348)
- Titus Josef Brinker (ORCID: https://orcid.org/0000-0002-3620-5919)
Institutions
- German Cancer Research Center (DE)
- Johannes Gutenberg University Mainz (DE)
- Heidelberg University (DE)
- University Hospital Heidelberg (DE)
- University Medical Center of the Johannes Gutenberg University Mainz (DE)
- Fresenius (Germany) (DE)
- National Center for Tumor Diseases (DE)
- Universitätsklinikum Aachen (DE)
- University Medical Centre Mannheim (DE)
- Technische Universität Dresden (DE)
- RWTH Aachen University (DE)
Publication Details
- Journal
- Nature Communications
- Published
- 2025-02-01
- DOI
- https://doi.org/10.1038/s41467-024-55631-x
- Citations
- 29
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 18.98
Funders
- Pfizer
- AstraZeneca
- National Institute for Health and Care Research
- Department of Health and Social Care
- European Commission
- Deutscher Akademischer Austauschdienst
- Deutsche Forschungsgemeinschaft
- Bundesministerium für Bildung und Forschung
- Bundesministerium für Gesundheit
- Eisai
- Deutsche Krebshilfe
- RWTH Aachen University
- Gemeinsame Bundesausschuss
- National Institutes of Health
- HORIZON EUROPE Framework Programme
- Leeds Biomedical Research Centre
- Daiichi Sankyo Europe