Prompt-Injection Defences on a Small Local Agent: What Works, What It Costs, and How Optimistic Development Numbers Are
An AI agent that reads email, files and web pages for a user can be hijacked by instructions an attacker plants in that data: indirect prompt injection. Published defences are mostly tested on large hosted models. I test defences on a small model that runs locally, Qwen3.5-4B, on the AgentDojo benchmark. A safety rule in the system prompt had no measurable effect on this model, so I built a defence in code, one measured change at a time. The result, plan-then-gate, combines published ideas: before reading any data, the model lists the actions the user asked for. Any state-changing action not on that list, and any action that moves money, must pass a separate check that sees only the user's request and the proposed call, not the tool output the agent has read. I then evaluated it under a protocol fixed before the final runs, on 158 attacked test cases (78 never run before) and 47 normal tasks. Attack success fell from 38% to 11% under AgentDojo's standard attack, and from 46% to 14% under a stronger one. Two of AgentDojo's built-in defences, the tool filter and a prompt-injection classifier, reach 18% and 6% attack success but complete 27 and 29 percentage points fewer normal tasks than plan-then-gate. A third, repeating the user's prompt, keeps utility but leaves attack success at 19%. Plan-then-gate has costs: it wrongly refuses a requested action in 4 of 47 normal tasks. Its effect on normal-task completion against no defence is −3 percentage points, but with only 47 tasks the 95% interval runs from −12 to +4, so the effect is inconclusive. Attacks that need no state-changing action, such as getting the agent to visit a link, pass it almost untouched. I also report what building the defence taught about measuring one: the benchmark's harness undercounted attacks, and an LLM judge rewarded the defence's own warnings.
Authors
- Syed Muhammad Hassan Raza (ORCID: https://orcid.org/0000-0002-5267-7078)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-30
- DOI
- https://doi.org/10.5281/zenodo.23811220
- Primary Topic
- Adversarial Robustness in Machine Learning
- Type
- preprint