Reproducible Is Not Correct: Reflexive Gates and the Self-Descriptions We Ship to Language Models
Software increasingly ships descriptions of itself intended for a language model rather than a person: specifications, constrained-decoding grammars, JSON schemas, vendored context files, and the identity a server states to its host at handshake time. These artifacts are neither compiled nor executed against the thing they describe. The gate deployed over them, almost universally, is reflexive: regenerate the artifact and fail on a difference from the committed copy. We observe that such a gate establishes determinism and cannot, even in principle, establish truth — and that it is nonetheless sound wherever it is deployed, because the artifacts it guards are mechanically derived from their subject, and a derived artifact can be stale but never wrong. That precondition is never stated and never checked. When a generator retypes a fact instead of deriving it, the same gate is vacuous, and it looks identical in continuous integration. We measure what this costs. A census of all 24,131 servers listed on the Model Context Protocol registry, covering the 7,272 unique npm packages among them, finds that of the 5,052 packages whose self-reported identity can be resolved statically, 89.6% retype it rather than derive it, and 41.1% state a version that once tracked the package and has since been left behind — corroborated against each package's own release history, with independently versioned cases excluded. We add six instances across four independent third-party compilers, four of them verified by execution, and a depth study of one production language whose four self-descriptions were all wrong on first check. We claim none of the underlying concepts, all of which are established; we report the precondition, the artifact class, and the measurement. The subject system was substantially co-authored with language models, and we discuss what that implies for which guards matter.
Authors
- Chan Meng (ORCID: https://orcid.org/0009-0002-1446-778X)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-09-25
- DOI
- https://doi.org/10.5281/zenodo.22958002
- Primary Topic
- Software Engineering Research
- Type
- preprint