Transient Low-Order Cues and Compositional Representation Onset: A Controlled Finite-Field Study

What controls the accessibility of a compositional prediction rule when the target conditional and total predictable information are unchanged? We study this question in a finite-field, parent-exposed Transformer task by varying the early joint input sampling distribution, then returning all cue-bearing conditions to uniform full support at 5M exposures. The interventions preserve the target conditional, noise, parent marginals, and total pairwise predictable information, while changing lower-order predictive cues and joint distribution structure together. In eight prospectively specified design cells sharing only two distinct initialization seeds, the registered cue levels C_single = 0, 0.016, and 0.065092 yielded verified onset by 10M in 1/8, 6/8, and 8/8 trajectories, respectively. Verified onset requires normalized full-grid risk R < 0.9 and layer-6 linear-probe accuracy > 0.5 at two successive registered probe checkpoints. Mean full-grid normalized KL risks at 10M were 0.432743, 0.110766, and 0.005181. All 13 cue-bearing trajectories already threshold-positive at 5M remained positive at both 6M and 10M under uniform continuation. The registered Page-style label-permutation reference has an exact upper-tail fraction of 16/6⁸, but the frozen protocol does not establish the joint label exchangeability required for a calibrated significance p-value. We therefore report bounded computational observations, not a statistically confirmed population effect or identified causal mediator. This case motivates examining the joint sampling law and the time course of representation formation alongside information-budget summaries. The accompanying limited verification supplement provides the eight reported onset triples, a standalone reference-count verifier, and analytical checks of the finite-field controls. The original training trajectories and checkpoints remain in the local archive.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-06
DOI
https://doi.org/10.5281/zenodo.24011962
Primary Topic
Neural Networks and Applications
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Transient Low-Order Cues and Compositional Representation Onset: A Controlled Finite-Field Study

Yongjing Gao
Zenodo (CERN European Organization for Nuclear Research)
Neural Networks and Applications
preprint

Transient Low-Order Cues and Compositional Representation Onset: A Controlled Finite-Field Study

Yongjing Gao
preprint en

Abstract

What controls the accessibility of a compositional prediction rule when the target conditional and total predictable information are unchanged? We study this question in a finite-field, parent-exposed Transformer task by varying the early joint input sampling distribution, then returning all cue-bearing conditions to uniform full support at 5M exposures. The interventions preserve the target conditional, noise, parent marginals, and total pairwise predictable information, while changing lower-order predictive cues and joint distribution structure together. In eight prospectively specified design cells sharing only two distinct initialization seeds, the registered cue levels C_single = 0, 0.016, and 0.065092 yielded verified onset by 10M in 1/8, 6/8, and 8/8 trajectories, respectively. Verified onset requires normalized full-grid risk R < 0.9 and layer-6 linear-probe accuracy > 0.5 at two successive registered probe checkpoints. Mean full-grid normalized KL risks at 10M were 0.432743, 0.110766, and 0.005181. All 13 cue-bearing trajectories already threshold-positive at 5M remained positive at both 6M and 10M under uniform continuation. The registered Page-style label-permutation reference has an exact upper-tail fraction of 16/6⁸, but the frozen protocol does not establish the joint label exchangeability required for a calibrated significance p-value. We therefore report bounded computational observations, not a statistically confirmed population effect or identified causal mediator. This case motivates examining the joint sampling law and the time course of representation formation alongside information-budget summaries. The accompanying limited verification supplement provides the eight reported onset triples, a standalone reference-count verifier, and analytical checks of the finite-field controls. The original training trajectories and checkpoints remain in the local archive.

Zenodo (CERN European Organization for Nuclear Research)
Neural Networks and Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon