The Cell, Not the Call: A Preregisterable Audit Protocol for Repeated-Call Inference in Behavioural LLM Experiments
Abstract. Repeated calls to a hosted large language model are routinely analysed as independent participants. Concurrent statistical work has diagnosed that dependence and supplied estimators for it. What it does not supply is the operational layer: the registered rules that make provider outputs auditable before any estimator runs. This tutorial supplies that layer as a five-module, preregisterable audit protocol covering the inferential unit, per-model stimulus calibration, response taxonomy, endpoint stability, and a reviewer-verifiable audit trail. We report its first full preregistered execution, a four-model moral-judgement experiment with 11,200 main trials and 40,871 logged records. Every model tripped something. Stimuli written to be ambiguous sat at ceiling for two endpoints, and the registered calibration fallback fired for every model, logged as a disclosed deviation. One model needed a registered recovery route to reach 86.0% parseable on severity, and classified 34.8% of supposedly inert control items as concern-raising. Three models passed the stability gate, two while moving 0.55 and 0.45 on the response scale, and the fourth failed at r = -.135 while moving 0.10. In simulation, naive call-level analysis inflated false-positive rates two- to six-fold under cell heterogeneity, and aggregating to cells without modelling dispersion reproduced that rate exactly, as binomial sufficiency requires. Preregistration templates, a reviewer checklist, simulation code, and hash-locked artefacts accompany the paper: publicly reproducible at the tutorial layer, and reviewer-verifiable at the worked case through a controlled-access route. One worked case cannot license general claims. It can show what a protocol catches when it is registered in advance. Deposit contents. This Zenodo version contains the preprint PDF only. Related OSF registration: https://osf.io/6m3sk/. Related preprints: Moral concern is a response regime, https://doi.org/10.5281/zenodo.22086437; No variance, no verdict, https://doi.org/10.5281/zenodo.21879504. Version note. Preprint v4 updates the public preprint from v3.1 to the current manuscript. Rewritten abstract and introduction, updated worked-case glance, a provenance note on the August 2026 snapshot, and citations to the now-public Paper 1 and Paper 3 preprints.
Authors
- Emile Boullineau
- José Daniel Muñoz Arciniegas
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-08-24
- DOI
- https://doi.org/10.5281/zenodo.22087214
- Primary Topic
- Neural dynamics and brain function
- Type
- preprint