When Does Context Compaction Pay in Long-Horizon Agents?

Abstract Compacting an agent's history shortens later requests but can change the work needed to finish its task. We ask when early compaction pays after counting that change: only when a fixed-trace savings budget exceeds the cost of the changed work. We measure both with matched-state continuations to termination on nine long Terminal-Bench tasks, complete runs in two agent runtimes, and replays on a dedicated GPU deployment. At a 64K threshold, gpt-6.1-sol's budget was USD 0.17 per state, yet synchronous compaction (Sync) cost USD 0.26 more with measured cache reads. Added requests, mostly reads, exhausted the budget; the excess persisted across prompt, task, and runtime controls. With GLM-5.3-Flash, compaction saved. Cache prices set the budget: for our nine-task mix of complete 64K Sol runs, Sync would cost less only above a cache-read price of 34% of the uncached price, against a listed 5%. At one common price vector, savings appeared only where much work remained. On a dedicated GLM deployment, replayed Sync sessions took 30% less mean time than sessions keeping the full history at light load (3% with the work fixed) and at least 29% less at heavy load. At 102 concurrent sessions without a host-memory cache tier, waiting sessions' evicted prefixes had to be computed again, and Sync's work with the history restored took more than twice as long. We release an offline calibrator mapping complete-task usage and deployment prices to policy costs, price crossovers, and selection headroom.

Authors

Publication Details

Journal
Research Square
Published
2026-10-30
DOI
https://doi.org/10.21203/rs.3.rs-11487641/v2
Primary Topic
Cloud Computing and Resource Management
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

When Does Context Compaction Pay in Long-Horizon Agents?

Yuyao Wang
1 links
Research Square
Cloud Computing and Resource Management
preprint

When Does Context Compaction Pay in Long-Horizon Agents?

Yuyao Wang
preprint en

Abstract

Abstract Compacting an agent's history shortens later requests but can change the work needed to finish its task. We ask when early compaction pays after counting that change: only when a fixed-trace savings budget exceeds the cost of the changed work. We measure both with matched-state continuations to termination on nine long Terminal-Bench tasks, complete runs in two agent runtimes, and replays on a dedicated GPU deployment. At a 64K threshold, gpt-6.1-sol's budget was USD 0.17 per state, yet synchronous compaction (Sync) cost USD 0.26 more with measured cache reads. Added requests, mostly reads, exhausted the budget; the excess persisted across prompt, task, and runtime controls. With GLM-5.3-Flash, compaction saved. Cache prices set the budget: for our nine-task mix of complete 64K Sol runs, Sync would cost less only above a cache-read price of 34% of the uncached price, against a listed 5%. At one common price vector, savings appeared only where much work remained. On a dedicated GLM deployment, replayed Sync sessions took 30% less mean time than sessions keeping the full history at light load (3% with the work fixed) and at least 29% less at heavy load. At 102 concurrent sessions without a host-memory cache tier, waiting sessions' evicted prefixes had to be computed again, and Sync's work with the history restored took more than twice as long. We release an offline calibrator mapping complete-task usage and deployment prices to policy costs, price crossovers, and selection headroom.

Research Square
Cloud Computing and Resource Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon