Probe-Distribution Confounding in Induction-Head Identification

This preprint studies probe-distribution confounding in induction-head identification. A fixed-period copy-task probe makes the induction target coincide with a constant positional offset, so a position-counting shortcut can receive the same prefix-matching score as a content-based induction mechanism. Using synthetic two-layer transformers, out-of-distribution repeat-period evaluation, content-blind reference models, attention diagnostics, and zero-ablation, the study shows that a high induction score can coexist with a non-generalizing positional solution. The scope is limited to randomly initialized transformers trained on synthetic copy tasks. The current archival record does not preserve the exact vocabulary size, optimizer, learning rate, or total training-step configuration; those values are not fabricated and must be recovered before claiming complete independent reproduction. Code and documentation are available at https://github.com/Zoe4370/TransInterp.