The Silent Saboteur: Imperceptible Adversarial Attacks against Black-Box Retrieval-Augmented Generation Systems
We explore adversarial attacks against retrievalaugmented generation (RAG) systems to identify their vulnerabilities.We focus on generating human-imperceptible adversarial examples and introduce a novel imperceptible retrieveto-generate attack against RAG.This task aims to find imperceptible perturbations that retrieve a target document, originally excluded from the initial top-k candidate set, in order to influence the final answer generation.To address this task, we propose ReGENT, a reinforcement learning-based framework that tracks interactions between the attacker and the target RAG and continuously refines attack strategies based on relevance-generation-naturalness rewards.Experiments on newly constructed factual and non-factual question-answering benchmarks demonstrate that ReGENT significantly outperforms existing attack methods in misleading RAG systems with small imperceptible text perturbations.
Authors
- Ruqing Zhang (ORCID: https://orcid.org/0000-0003-4294-2541)
- Xueqi Cheng (ORCID: https://orcid.org/0000-0002-5201-8195)
- Maarten de Rijke (ORCID: https://orcid.org/0000-0002-1086-0202)
- Jianming Lv (ORCID: https://orcid.org/0000-0003-3691-4755)
- Jiafeng Guo (ORCID: https://orcid.org/0000-0002-9509-8674)
- Yuan Liu (ORCID: https://orcid.org/0000-0002-6326-1270)
- Hongru Song
Publication Details
- Published
- 2025-01-01
- DOI
- https://doi.org/10.18653/v1/2025.findings-acl.717
- Primary Topic
- Adversarial Robustness in Machine Learning
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- European Commission
- National Natural Science Foundation of China
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek
- HORIZON EUROPE Framework Programme
- National Key Research and Development Program of China