The Cell, Not the Call: A Preregisterable Audit Protocol for Repeated-Call Inference in Behavioural LLM Experiments

Abstract. Repeated calls to a hosted large language model are routinely analysed as independent participants. Concurrent statistical work has diagnosed that dependence and supplied estimators for it. What it does not supply is the operational layer: the registered rules that make provider outputs auditable before any estimator runs. This tutorial supplies that layer as a five-module, preregisterable audit protocol covering the inferential unit, per-model stimulus calibration, response taxonomy, endpoint stability, and a reviewer-verifiable audit trail. We report its first full preregistered execution, a four-model moral-judgement experiment with 11,200 main trials and 40,871 logged records. Every model tripped something. Stimuli written to be ambiguous sat at ceiling for two endpoints, and the registered calibration fallback fired for every model, logged as a disclosed deviation. One model needed a registered recovery route to reach 86.0% parseable on severity, and classified 34.8% of supposedly inert control items as concern-raising. Three models passed the stability gate, two while moving 0.55 and 0.45 on the response scale, and the fourth failed at r = -.135 while moving 0.10. In simulation, naive call-level analysis inflated false-positive rates two- to six-fold under cell heterogeneity, and aggregating to cells without modelling dispersion reproduced that rate exactly, as binomial sufficiency requires. Preregistration templates, a reviewer checklist, simulation code, and hash-locked artefacts accompany the paper: publicly reproducible at the tutorial layer, and reviewer-verifiable at the worked case through a controlled-access route. One worked case cannot license general claims. It can show what a protocol catches when it is registered in advance. Deposit contents. This Zenodo version contains the preprint PDF only. Related OSF registration: https://osf.io/6m3sk/. Related preprints: Moral concern is a response regime, https://doi.org/10.5281/zenodo.22086437; No variance, no verdict, https://doi.org/10.5281/zenodo.21879504. Version note. Preprint v4 updates the public preprint from v3.1 to the current manuscript. Rewritten abstract and introduction, updated worked-case glance, a provenance note on the August 2026 snapshot, and citations to the now-public Paper 1 and Paper 3 preprints.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-08-24
DOI
https://doi.org/10.5281/zenodo.22087214
Primary Topic
Neural dynamics and brain function
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

The Cell, Not the Call: A Preregisterable Audit Protocol for Repeated-Call Inference in Behavioural LLM Experiments

Emile Boullineau, José Daniel Muñoz Arciniegas
Zenodo (CERN European Organization for Nuclear Research)
Neural dynamics and brain function
preprint

The Cell, Not the Call: A Preregisterable Audit Protocol for Repeated-Call Inference in Behavioural LLM Experiments

Emile Boullineau, José Daniel Muñoz Arciniegas
preprint en

Abstract

Abstract. Repeated calls to a hosted large language model are routinely analysed as independent participants. Concurrent statistical work has diagnosed that dependence and supplied estimators for it. What it does not supply is the operational layer: the registered rules that make provider outputs auditable before any estimator runs. This tutorial supplies that layer as a five-module, preregisterable audit protocol covering the inferential unit, per-model stimulus calibration, response taxonomy, endpoint stability, and a reviewer-verifiable audit trail. We report its first full preregistered execution, a four-model moral-judgement experiment with 11,200 main trials and 40,871 logged records. Every model tripped something. Stimuli written to be ambiguous sat at ceiling for two endpoints, and the registered calibration fallback fired for every model, logged as a disclosed deviation. One model needed a registered recovery route to reach 86.0% parseable on severity, and classified 34.8% of supposedly inert control items as concern-raising. Three models passed the stability gate, two while moving 0.55 and 0.45 on the response scale, and the fourth failed at r = -.135 while moving 0.10. In simulation, naive call-level analysis inflated false-positive rates two- to six-fold under cell heterogeneity, and aggregating to cells without modelling dispersion reproduced that rate exactly, as binomial sufficiency requires. Preregistration templates, a reviewer checklist, simulation code, and hash-locked artefacts accompany the paper: publicly reproducible at the tutorial layer, and reviewer-verifiable at the worked case through a controlled-access route. One worked case cannot license general claims. It can show what a protocol catches when it is registered in advance. Deposit contents. This Zenodo version contains the preprint PDF only. Related OSF registration: https://osf.io/6m3sk/. Related preprints: Moral concern is a response regime, https://doi.org/10.5281/zenodo.22086437; No variance, no verdict, https://doi.org/10.5281/zenodo.21879504. Version note. Preprint v4 updates the public preprint from v3.1 to the current manuscript. Rewritten abstract and introduction, updated worked-case glance, a provenance note on the August 2026 snapshot, and citations to the now-public Paper 1 and Paper 3 preprints.

Zenodo (CERN European Organization for Nuclear Research)
Peace, Justice and strong institutions
Neural dynamics and brain function
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

The Cell, Not the Call: A Preregisterable Audit Protocol for Repeated-Call Inference in Behavioural LLM Experiments — Emile Boullineau, José Daniel Muñoz Arciniegas · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS