Control Is All You Need: Secure Before It Acts
A tool call an agent proposes and a tool call an agent should be allowed to execute are not necessarily the same thing. The security decision that sits between those two moments, immediately before execution, has received far less attention than prompts, outputs, or agent trajectories. This work explores pre-execution action judgment as a distinct security control for AI agents: given a proposed action and its available context, should it execute? I built saroku-guard, a 184M-parameter DeBERTa-v3 classifier specifically for this decision, and saroku, an open-source runtime enforcement layer that places the decision directly in the agent's execution path. The interface between decision and enforcement is then formalized as the Action Safety Protocol (ASP). Evaluating the approach required a benchmark that didn't exist, so I built ASP-Bench: a set of pre-execution agent actions spanning 16 primary domains, with safe/unsafe labels, violation categories, and severity. Benchmarking saroku-guard alongside existing guard, moderation, and agent-safety models on a frozen private holdout using each model's documented native input format, saroku-guard achieved 97.9% unsafe recall, 2.9% safe-action blocking, and 8.6ms p99 latency, nearly 5x faster than AgentDoG's 42.2ms. The strongest peer on unsafe recall, AgentDoG, reached 99.7% unsafe recall but blocked 72.1% of safe actions, exposing the central trade-off for inline agent security: catching unsafe actions without blocking legitimate work. The result is a three-part system: ASP as the decision contract, saroku-guard as the PDP, and saroku as the PEP.
Authors
- Karan Arora
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-09-28
- DOI
- https://doi.org/10.5281/zenodo.23023309
- Primary Topic
- Adversarial Robustness in Machine Learning
- Type
- preprint