What Next-Token Surprisal Can and Cannot See: A Registered Null Result on Measuring Narrative Reorganization in a Language Model

We tested whether semantic reorganization, the “reveal” moment in which new information reframes previously read material, leaves a measurable trace in per-token surprisal when a large language model (Gemma 4 31B-it) reads short narrative texts. Ten matched stimulus pairs were constructed with byte-identical openings diverging into a reveal continuation (reframing the opening) or a neutral continuation (extending it without reframing). Predictions were registered before any data collection. The registered hypothesis: elevated surprisal in the post-reveal region. This was not supported by the data. Post-region contrasts averaged −1.12 bits (negative in seven of ten pairs): reveal continuations were cheaper to read than neutral ones. Token-level inspection identifies the mechanism: reveal continuations, designed to re-read the opening, re-quote its content words, and in-context repetition collapses their surprisal by one to three orders of magnitude (e.g., 33.1 → 0.03 bits), swamping any reorganization-related elevation. We conclude that next-token surprisal is dominated by lexical novelty and recency and is therefore structurally unsuited to detecting reorganization in designs where reorganization expresses itself as re-reading. We describe a successor measure that is immune to the repetition discount by construction, and register its prediction. The measure is hidden-state distance on identical quoted-back text. The null is reported in full, per the research program’s standing practice (Bruyn, 2026a).