SAFE-SCALE: Unified Sub-Quadratic Architectures, Adaptive Test-Time Compute, and Mechanistic Steering for Long-Horizon Autonomous Agents

Long-horizon autonomous agents must keep execution histories that run to millions of tokens, spend test-time compute on deliberate multi-step search, and resist adversarial content arriving through tool outputs. These requirements conflict. Softmax attention incurs compute quadratic in trajectory length and a key-value (KV) cache that grows linearly per layer; tree search multiplies that state by the branching factor; and external guardrail models add serial latency at every tool boundary. We call this tension the Agentic Trilemma of capability, cost, and safety. We present SAFE-SCALE, a backbone-to-inference framework that resolves it by co-design rather than composition. (i) A hybrid backbone interleaves Gated DeltaNet and Mamba-2 layers with block-sparse attention at a 7:1 ratio, distilled from a dense 70B teacher, reducing the per-branch fork state to a constant ≈0.29 GB of recurrent state plus a shared, sparsely read attention cache. (ii) A difficulty-aware router invokes Process Reward Model (PRM)-guided speculative tree search only on high-uncertainty steps, exploiting constant-size state snapshots for O(1) fork and rollback. (iii) Sparse Autoencoders trained jointly on residual streams and SSM states expose monosemantic features for prompt injection, goal hijacking, and unauthorized tool invocation; a fused linear probe clamps these features and prunes compromised branches before any tool call is emitted. On SWE-bench Verified, RULER at 1M tokens, and AgentDojo, SAFE-SCALE targets 61.8% resolution, 84.1 RULER-1M accuracy, and an attack success rate of 2.9% (versus 24.7% undefended), while reducing peak VRAM by 5.3× and end-to-end latency by 2.7× relative to a dense 70B Transformer running the same search budget.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-21
DOI
https://doi.org/10.5281/zenodo.24414228
Primary Topic
Adversarial Robustness in Machine Learning
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

SAFE-SCALE: Unified Sub-Quadratic Architectures, Adaptive Test-Time Compute, and Mechanistic Steering for Long-Horizon Autonomous Agents

Prateek Dutta
Zenodo (CERN European Organization for Nuclear Research)
Adversarial Robustness in Machine Learning
preprint

SAFE-SCALE: Unified Sub-Quadratic Architectures, Adaptive Test-Time Compute, and Mechanistic Steering for Long-Horizon Autonomous Agents

Prateek Dutta
preprint en

Abstract

Long-horizon autonomous agents must keep execution histories that run to millions of tokens, spend test-time compute on deliberate multi-step search, and resist adversarial content arriving through tool outputs. These requirements conflict. Softmax attention incurs compute quadratic in trajectory length and a key-value (KV) cache that grows linearly per layer; tree search multiplies that state by the branching factor; and external guardrail models add serial latency at every tool boundary. We call this tension the Agentic Trilemma of capability, cost, and safety. We present SAFE-SCALE, a backbone-to-inference framework that resolves it by co-design rather than composition. (i) A hybrid backbone interleaves Gated DeltaNet and Mamba-2 layers with block-sparse attention at a 7:1 ratio, distilled from a dense 70B teacher, reducing the per-branch fork state to a constant ≈0.29 GB of recurrent state plus a shared, sparsely read attention cache. (ii) A difficulty-aware router invokes Process Reward Model (PRM)-guided speculative tree search only on high-uncertainty steps, exploiting constant-size state snapshots for O(1) fork and rollback. (iii) Sparse Autoencoders trained jointly on residual streams and SSM states expose monosemantic features for prompt injection, goal hijacking, and unauthorized tool invocation; a fused linear probe clamps these features and prunes compromised branches before any tool call is emitted. On SWE-bench Verified, RULER at 1M tokens, and AgentDojo, SAFE-SCALE targets 61.8% resolution, 84.1 RULER-1M accuracy, and an attack success rate of 2.9% (versus 24.7% undefended), while reducing peak VRAM by 5.3× and end-to-end latency by 2.7× relative to a dense 70B Transformer running the same search budget.

Zenodo (CERN European Organization for Nuclear Research)
Adversarial Robustness in Machine Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon