Hermeticity Is Not Binary: What Resetting Catches, What It Cannot, and What It Costs

Agents that crystallize session code into reusable skills need a promotion gate: may this candidate be lifted out of its session and called again later? The standard check, running the candidate again in a fresh, hermetic environment, treats hermeticity as binary. We show it is a family of choices, and that what you reset determines what you can observe. Using a two-arm repro-run harness over three reset axes (namespace, filesystem, network), we show by ablation that the process boundary itself contributes no evidence: resetting a declared namespace is a null operation, while filesystem and network resets are the detectors. We then prove a structural limit: dependencies on ambient files, assumed to pre-exist rather than created in-session, have no detecting configuration under a hermetic two-arm comparison; they exit the measured population instead of appearing in it, and no larger corpus repairs this. Finally we price the opposite error from our own harness's incident log: over-resetting, in our case discarding a benign font cache while isolating the HOME directory, produced 79 false timeouts and 18 changed verdicts. This is the same failure mode the flaky-test literature calls brittles, here characterized for the first time in hermetic validation of promotion candidates and framed as a measurement-validity error rather than flakiness. The accompanying 3.98x slowdown confirms a known isolation-cost corridor. From these symmetric failures we derive a granularity principle: reset scope is chosen per cache, by the provenance of its contents. Detectors, corpus, and every reported number ship as reproducible artifacts, with the promotion verifier making zero model calls.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-08-26
DOI
https://doi.org/10.5281/zenodo.22128319
Primary Topic
Software Engineering Research
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Hermeticity Is Not Binary: What Resetting Catches, What It Cannot, and What It Costs

Nikolay Krashnikov
Zenodo (CERN European Organization for Nuclear Research)
Software Engineering Research
preprint

Hermeticity Is Not Binary: What Resetting Catches, What It Cannot, and What It Costs

Nikolay Krashnikov
preprint en

Abstract

Agents that crystallize session code into reusable skills need a promotion gate: may this candidate be lifted out of its session and called again later? The standard check, running the candidate again in a fresh, hermetic environment, treats hermeticity as binary. We show it is a family of choices, and that what you reset determines what you can observe. Using a two-arm repro-run harness over three reset axes (namespace, filesystem, network), we show by ablation that the process boundary itself contributes no evidence: resetting a declared namespace is a null operation, while filesystem and network resets are the detectors. We then prove a structural limit: dependencies on ambient files, assumed to pre-exist rather than created in-session, have no detecting configuration under a hermetic two-arm comparison; they exit the measured population instead of appearing in it, and no larger corpus repairs this. Finally we price the opposite error from our own harness's incident log: over-resetting, in our case discarding a benign font cache while isolating the HOME directory, produced 79 false timeouts and 18 changed verdicts. This is the same failure mode the flaky-test literature calls brittles, here characterized for the first time in hermetic validation of promotion candidates and framed as a measurement-validity error rather than flakiness. The accompanying 3.98x slowdown confirms a known isolation-cost corridor. From these symmetric failures we derive a granularity principle: reset scope is chosen per cache, by the provenance of its contents. Detectors, corpus, and every reported number ship as reproducible artifacts, with the promotion verifier making zero model calls.

Zenodo (CERN European Organization for Nuclear Research)
Quality Education
Software Engineering Research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.