Blinded, contract-verified reproduction of a published computational step by a local 9B language model: architecture and case study
Objective: To test whether an automated research system running a small, locally hosted language model can reproduce one computational step of a published work under rules fixed before execution, without any agent having access to the published answer. Methods: We describe an architecture in which AI agents build, run, and review the work; an automatic judge requires a signed act from the human owner for any change of criteria and two external reviews to close each block of work; and the reference answer stays in human custody, out of the agents' reach. The selected step was the scientific energy gain of a National Ignition Facility shot, a ratio of two energies. The evaluated model was Qwen3.5-9B, quantized to 4 bits and run on CPU without network access. Each attempt had two turns: the model wrote code, the system executed it once in Python, and the model delivered a JSON object in the contract format. Seven automatic checks (V1–V7) decided acceptance, under a global limit of 30 min. Results: The primary run ended inconclusively: two outputs were rejected for a JSON-object format error and the third attempt was stopped by the time limit, without an accepted output. Subsequent analysis showed misplaced keys and no sign of truncated output. A secondary run, selected after the failure and registered before execution, repeated the required format as an explicit skeleton and was accepted on its first attempt, with no rejections and no Python errors. The blinded text was frozen before the comparison, which only the owner performed; its outcome is not part of this body. Conclusion: The architecture recorded, end to end, an inconclusive primary run and an accepted secondary run, kept separate. The data do not establish scientific correctness, an isolated causal effect of the instruction, independent replication, or general investigative ability. Files: English version (version of record), Portuguese version with the same scientific content, and supplementary material. Code, permitted records, and provenance: https://github.com/JorgeYuriDj/pesquisa-automatizada-m1 This study reproduces one computational step of Wurzel and Hsu (2025), Physics of Plasmas 32, 112106, doi:10.1063/5.0297357. The scientific merit of the original work belongs to its authors, who did not participate in, review, or endorse this study. Not peer reviewed.
Authors
- Jorge Yuri dos Santos Correa (ORCID: https://orcid.org/0009-0006-7378-4098)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-02
- DOI
- https://doi.org/10.5281/zenodo.23891978
- Primary Topic
- Scientific Computing and Data Management
- Type
- preprint