Execution Information and Estimation Timing for Coding Agent ETA: An Empirical Study and a Delivery-Aware Framework

Coding agents produce execution histories that can inform estimates of their remaining runtime. Designing a useful ETA service requires understanding which information improves forecasts, when it becomes useful, and how prediction latency affects delivery. We study these questions through same-anchor information comparisons, a four-model timing study, and frozen-trace replay, and implement a framework linking observable execution prefixes to delivered countdowns. Our retrospective synthesis evaluates raw error, signed bias, task discrimination, and display coverage. Richer execution packages reduce raw error in several settings: at 60 seconds, mean absolute error falls from 396.0 to 189.7 seconds on eleven separate confirmation tasks. The benefit varies with estimation time and execution environment. Canonical replay shows that later error reductions can primarily reflect less overestimation, without an established improvement in task discrimination. Across four models, absolute error decreases at later stages, while relative accuracy is not uniformly monotonic. Delivery analysis further shows why improved point forecasts need an evaluated update policy to benefit a displayed countdown. We contribute a joint empirical characterization of information and timing, a measurement approach separating error reduction from bias and discrimination, and an auditable ETA framework. The evidence concerns the studied information packages and trajectories; originally frozen calibrated primary tests remain inconclusive. Preprint; not peer reviewed. Version 1.7. Code and companion numerical and methods materials: https://github.com/kinchinglam66-alt/codex-progress-hud/releases/tag/research-release-2026-10-24

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-24
DOI
https://doi.org/10.5281/zenodo.23613203
Primary Topic
Software Engineering Research
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Execution Information and Estimation Timing for Coding Agent ETA: An Empirical Study and a Delivery-Aware Framework

Kin Ching Lam
Zenodo (CERN European Organization for Nuclear Research)
Software Engineering Research
preprint

Execution Information and Estimation Timing for Coding Agent ETA: An Empirical Study and a Delivery-Aware Framework

Kin Ching Lam
preprint en

Abstract

Coding agents produce execution histories that can inform estimates of their remaining runtime. Designing a useful ETA service requires understanding which information improves forecasts, when it becomes useful, and how prediction latency affects delivery. We study these questions through same-anchor information comparisons, a four-model timing study, and frozen-trace replay, and implement a framework linking observable execution prefixes to delivered countdowns. Our retrospective synthesis evaluates raw error, signed bias, task discrimination, and display coverage. Richer execution packages reduce raw error in several settings: at 60 seconds, mean absolute error falls from 396.0 to 189.7 seconds on eleven separate confirmation tasks. The benefit varies with estimation time and execution environment. Canonical replay shows that later error reductions can primarily reflect less overestimation, without an established improvement in task discrimination. Across four models, absolute error decreases at later stages, while relative accuracy is not uniformly monotonic. Delivery analysis further shows why improved point forecasts need an evaluated update policy to benefit a displayed countdown. We contribute a joint empirical characterization of information and timing, a measurement approach separating error reduction from bias and discrimination, and an auditable ETA framework. The evidence concerns the studied information packages and trajectories; originally frozen calibrated primary tests remain inconclusive. Preprint; not peer reviewed. Version 1.7. Code and companion numerical and methods materials: https://github.com/kinchinglam66-alt/codex-progress-hud/releases/tag/research-release-2026-10-24

Zenodo (CERN European Organization for Nuclear Research)
Software Engineering Research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon