The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents

Large Language Model (LLM) agents are increasingly being deployed as conversational assistants capable of performing complex realworld tasks through tool integration.This enhanced ability to interact with external systems and process various data sources, while powerful, introduces significant security vulnerabilities.In particular, indirect prompt injection attacks pose a critical threat, where malicious instructions embedded within external data sources can manipulate agents to deviate from user intentions.While existing defenses show promise, they struggle to maintain robust security while preserving task functionality.We propose a novel and orthogonal perspective that reframes agent security from preventing harmful actions to ensuring task alignment, requiring every agent action to serve user objectives.Based on this insight, we develop Task Shield, a test-time defense mechanism that systematically verifies whether each instruction and tool call contributes to user-specified goals.Through experiments on the Agent-Dojo benchmark, we demonstrate that Task Shield reduces attack success rates (2.07%) while maintaining high task utility (69.79%) on GPT-4o, significantly outperforming existing defenses in various real-world scenarios.

Authors

Publication Details

Published
2025-01-01
DOI
https://doi.org/10.18653/v1/2025.acl-long.1435
Citations
2
Primary Topic
Scheduling and Optimization Algorithms
Type
article
Field-Weighted Citation Impact
2.48

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents

Feiran Jia, Qin Xin, Anna Squicciarini, Tong Wu
2 citations
Scheduling and Optimization Algorithms
2.48
article

The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents

Feiran Jia, Qin Xin, Anna Squicciarini, Tong Wu
article en
2 citations

Abstract

Large Language Model (LLM) agents are increasingly being deployed as conversational assistants capable of performing complex realworld tasks through tool integration.This enhanced ability to interact with external systems and process various data sources, while powerful, introduces significant security vulnerabilities.In particular, indirect prompt injection attacks pose a critical threat, where malicious instructions embedded within external data sources can manipulate agents to deviate from user intentions.While existing defenses show promise, they struggle to maintain robust security while preserving task functionality.We propose a novel and orthogonal perspective that reframes agent security from preventing harmful actions to ensuring task alignment, requiring every agent action to serve user objectives.Based on this insight, we develop Task Shield, a test-time defense mechanism that systematically verifies whether each instruction and tool call contributes to user-specified goals.Through experiments on the Agent-Dojo benchmark, we demonstrate that Task Shield reduces attack success rates (2.07%) while maintaining high task utility (69.79%) on GPT-4o, significantly outperforming existing defenses in various real-world scenarios.

National Science Foundation
Gender equality
Openalex Percentile: Top 10%
Scheduling and Optimization Algorithms
2.48
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.