Runtime-State Watermarking in Transformers: A Controlled Protocol for Distinguishing Static Self-Simulation from Runtime-State Decodability

Whether a neural language model can report facts about its own computation is difficult to evaluate because an accurate report may be generated either from the realized runtime state or from a static simulation of the computation conditioned on the input and fixed parameters. We formalize this distinction and show that, under deterministic inference, an internal property of an unperturbed forward pass is not by itself sufficient to distinguish these mechanisms: the realized state is an exact deterministic function of the static input and parameters. We introduce the Exogenous State Watermark (ESW), a controlled intervention protocol in which an externally sampled perturbation is injected into an intermediate representation while the model's observable next-token output is audited for greedy invariance. For the isolated watermark bit, the static information set has a 50% Bayes-optimal chance bound under the stated independence assumptions. For the counterfactual target, we evaluate unperturbed-state and runtime representations empirically on GPT-2 (124M parameters; width 768). In ESW-1A, a planted directional watermark becomes increasingly decodable as training examples grow relative to representation dimension, reaching 88.4% held-out accuracy at N/d=6.51 (cos θ = 0.9236), while a one-dimensional control reaches 100.0%. In an exploratory Oracle Ladder diagnostic, differential signals peak at Layer 8 before downstream attenuation. In the subsequent multi-seed paired-counterfactual benchmark (ESW-2B), an external nonlinear probe of intermediate runtime differences reaches approximately 0.62–0.63 AUROC, while the tested unperturbed-state baseline remains close to chance (0.507 AUROC) and a sample-label decoupled control remains at 0.546 AUROC. The paired bootstrap difference between the joint and runtime-only models includes zero (ΔAUROC = +0.012, 95% CI [-0.023, +0.047]), demonstrating that adding baseline activations provides no statistically detectable advantage over the runtime perturbation alone. We interpret the experiments as empirical evidence for runtime-state-dependent decodability under the ESW intervention, not as evidence that GPT-2 possesses autonomous introspection, privileged self-report, consciousness, or subjective self-knowledge.