Contained or Escaped? Measuring Sandbox Escape and Auto-Reviewer Decisions in AI Coding Agents

Artificial intelligence (AI) coding agents now run shell commands in a default sandbox and must request permission, often screened by a large language model (LLM) auto-reviewer, before a command may use the network or write outside the workspace. Most prompt-injection benchmarks for coding agents assume unrestricted execution. We built SARD-Bench, 240 poisoned-workspace cases in four families and three stealth tiers, with a harness whose single tool makes the agent declare and justify the permissions it needs. We ran Claude Sonnet 5.5, GPT-5.1 and Llama 4 Maverick once per case (720 trajectories), executed the two-turn family in a macOS Seatbelt sandbox, and had a GPT-4o-mini reviewer judge every harmful command with and without the agent’s rationale. All labels were audited by hand in two passes. The two proprietary models ran the payload inside the sandbox in 28.3% of runs and requested the privileges to escape in 6.5%. Llama 4 Maverick escaped in 44.6%. Stealthier files raised in-sandbox execution from 10.6% to 50.6% but left escape rates unchanged. With the agent’s rationale, the reviewer blocked 126 of 138 escape requests, against 64 from the command alone. Claude Sonnet 5.5 returned provider refusals in 140 runs, and an indicator-matching scorer missed half of all escapes. Evaluations of sandboxed agents should report where a payload would run and audit their labels by hand.

Authors

Publication Details

Journal
Preprints.org
Published
2026-11-17
DOI
https://doi.org/10.20944/preprints202611.1647.v1
Primary Topic
Adversarial Robustness in Machine Learning
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Contained or Escaped? Measuring Sandbox Escape and Auto-Reviewer Decisions in AI Coding Agents

Deep Gandhi, Young Lee
Preprints.org
Adversarial Robustness in Machine Learning
preprint

Contained or Escaped? Measuring Sandbox Escape and Auto-Reviewer Decisions in AI Coding Agents

Deep Gandhi, Young Lee
preprint en

Abstract

Artificial intelligence (AI) coding agents now run shell commands in a default sandbox and must request permission, often screened by a large language model (LLM) auto-reviewer, before a command may use the network or write outside the workspace. Most prompt-injection benchmarks for coding agents assume unrestricted execution. We built SARD-Bench, 240 poisoned-workspace cases in four families and three stealth tiers, with a harness whose single tool makes the agent declare and justify the permissions it needs. We ran Claude Sonnet 5.5, GPT-5.1 and Llama 4 Maverick once per case (720 trajectories), executed the two-turn family in a macOS Seatbelt sandbox, and had a GPT-4o-mini reviewer judge every harmful command with and without the agent’s rationale. All labels were audited by hand in two passes. The two proprietary models ran the payload inside the sandbox in 28.3% of runs and requested the privileges to escape in 6.5%. Llama 4 Maverick escaped in 44.6%. Stealthier files raised in-sandbox execution from 10.6% to 50.6% but left escape rates unchanged. With the agent’s rationale, the reviewer blocked 126 of 138 escape requests, against 64 from the command alone. Claude Sonnet 5.5 returned provider refusals in 140 runs, and an indicator-matching scorer missed half of all escapes. Evaluations of sandboxed agents should report where a payload would run and audit their labels by hand.

Preprints.org
Adversarial Robustness in Machine Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon