What Next-Token Surprisal Can and Cannot See: A Registered Null Result on Measuring Narrative Reorganization in a Language Model

Version 2 (October 2026). An audit found that the registered model, Gemma 4 31B-it, reads plain narrative prose abnormally: about 9 bits per token, against about 3 bits for four other instruct models on the same texts, in two independent implementations. Its surprisal values therefore cannot serve as the measure, and the version 1 results are reported for the record but not interpreted. The measurement code was correct, and the July numbers reproduce exactly. The frozen stimuli were rerun on Qwen3.8 27B, as a declared deviation from the registration. Neither the trigger prediction nor the post-region hypothesis holds (mean trigger contrast −0.48 bits, mean post contrast −0.27 bits, each positive in four of ten pairs). The reversed effect, its repetition mechanism and the calibration account of version 1 are withdrawn. This version adds the audit, the second-model run, the corrected pipeline description and the frozen design document; the stimuli, July data and July code are unchanged. Version 1 remains available. The study tested whether semantic reorganization, the “reveal” moment in which new information reframes previously read material, leaves a measurable trace in per-token surprisal when a language model reads short narrative texts. Ten matched stimulus pairs had byte-identical openings diverging into a reveal or a neutral continuation, and predictions were registered before any data collection. The successor measure registered in version 1, hidden-state distance on identical quoted-back text, is kept unchanged. A plain-language introduction to the measures is given in Bruyn (2026), doi:10.5281/zenodo.23360915. The null is reported in full, per the research program’s standing practice.