Why Speech-Regex Action Supervisors Miss Live Output: An Empirical Evasion Taxonomy and Held-Out Structured Action Contract Evaluation
Autonomous agents frequently emit fabricated action claims (FACs)—asserting execution of real-world side effects (file writes, process terminations, configuration updates) that never occurred. While regular-expression speech supervisors achieve high sensitivity on synthetic templates, we show across N=120 pre-registered held-out prompts (prereg-supervision-v1 / v2) on Gemma-4-26B-A4B that strict surface speech-regex catches only 1/16 (6.25%) live ungrounded action claims due to four grammatical evasion modes: participial clauses, passive voice, markdown formatting, and unlisted verb lemmas. We evaluate Structured-Output Action Contracts (SOAC), binding action authority to typed tool frames verified against sandbox execution receipts in 6.82 microseconds mean latency with 0/40 false positives on conversational controls, and demonstrate that decode-time GBNF grammar constraints on the frame channel alone do not suppress prose assertion leakage (14.17% vs 12.50%, p = 0.7237), establishing that post-turn NLI receipt entailment is required as an active epistemic backstop.
Authors
- Sean Maxwell
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-03
- DOI
- https://doi.org/10.5281/zenodo.23901604
- Primary Topic
- Adversarial Robustness in Machine Learning
- Type
- preprint