Collapse, Read, Consolidate? Tracing a Jacobian-Lens Workspace Through Pythia Training

Recent work uses fitted Jacobian lenses to report a low-dimensional, output-readable “workspace” in trained language models, a set of directions in which task-relevant entities are linearly legible well before the final layer. This evidence comes almost entirely from fully trained checkpoints. We fit an independent Jacobian lens at each of 23 public training checkpoints (22 usable at 1.4B) of four Pythia models (70M–1.4B) and read three observables: the effective dimension of the lens-transported entity cone, the drift of lens coordinates away from the unembedding, and the rank of task answers and multi-hop intermediates in the per-layer readout. Under fixed onset criteria, the signatures arrive in Pythia in ordered stages. Effective dimension collapses first (steps 512–1,000), answer readability follows (steps 2,000–6,000), and multi-hop intermediate readability appears last (steps 9,000–13,000 in the 1B and deduplicated 1.4B runs) and only at sufficient scale. Answers precede intermediates under the top-10 criterion and under matched median-rank thresholds, but not under every criterion. In Pythia the early collapse is shared by the raw unembedding cone, and most of the lens-specific compression builds up after step 23,000. Natively measured induction heads form inside the collapse window, one stage before answers. Intermediates never become reliably readable at 410M, rebound once at 1B, and descend near-monotonically in the deduplicated 1.4B run. The separation of the 1B and deduplicated 1.4B trajectories in excess path length, a measure of reversals, does not survive leaving out single checkpoints. Without step 23,000 or step 43,000 (the one 1B rebound), its interval includes zero. Their endpoint separation in log10 median rank (0.87 [0.40, 1.16]) does not depend on these checkpoints. A second 1.4B run reproduces the lower level but not the near-monotone descent. On OLMo-1B, over the first 143,000 steps (about 10% of its run), the three stages arrive in the same order. The step windows, the lead of induction heads and the time course of the compression differ, and the intermediate median rank does not rebound, although only three checkpoints follow its onset. The scale account of consolidation is therefore a hypothesis within Pythia. We also document an anisotropy correction without which cross-model effective dimensions are incomparable, and a public Pythia checkpoint branch that contains the final weights and mimics a phase transition. This record contains the preprint and a supplementary archive with the per-checkpoint observable records behind every figure and table, the analysis and statistics code, and the figure-generation code. The consolidation test, the trajectory and onset checks, the OLMo comparison and the main figure recompute from the shipped records without a GPU. Code: https://github.com/tao-hpu/jspace-replication.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-17
DOI
https://doi.org/10.5281/zenodo.21696225
Primary Topic
Explainable Artificial Intelligence (XAI)
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Collapse, Read, Consolidate? Tracing a Jacobian-Lens Workspace Through Pythia Training

Tao An
2 links
Zenodo (CERN European Organization for Nuclear Research)
Explainable Artificial Intelligence (XAI)
preprint

Collapse, Read, Consolidate? Tracing a Jacobian-Lens Workspace Through Pythia Training

Tao An
preprint en

Abstract

Recent work uses fitted Jacobian lenses to report a low-dimensional, output-readable “workspace” in trained language models, a set of directions in which task-relevant entities are linearly legible well before the final layer. This evidence comes almost entirely from fully trained checkpoints. We fit an independent Jacobian lens at each of 23 public training checkpoints (22 usable at 1.4B) of four Pythia models (70M–1.4B) and read three observables: the effective dimension of the lens-transported entity cone, the drift of lens coordinates away from the unembedding, and the rank of task answers and multi-hop intermediates in the per-layer readout. Under fixed onset criteria, the signatures arrive in Pythia in ordered stages. Effective dimension collapses first (steps 512–1,000), answer readability follows (steps 2,000–6,000), and multi-hop intermediate readability appears last (steps 9,000–13,000 in the 1B and deduplicated 1.4B runs) and only at sufficient scale. Answers precede intermediates under the top-10 criterion and under matched median-rank thresholds, but not under every criterion. In Pythia the early collapse is shared by the raw unembedding cone, and most of the lens-specific compression builds up after step 23,000. Natively measured induction heads form inside the collapse window, one stage before answers. Intermediates never become reliably readable at 410M, rebound once at 1B, and descend near-monotonically in the deduplicated 1.4B run. The separation of the 1B and deduplicated 1.4B trajectories in excess path length, a measure of reversals, does not survive leaving out single checkpoints. Without step 23,000 or step 43,000 (the one 1B rebound), its interval includes zero. Their endpoint separation in log10 median rank (0.87 [0.40, 1.16]) does not depend on these checkpoints. A second 1.4B run reproduces the lower level but not the near-monotone descent. On OLMo-1B, over the first 143,000 steps (about 10% of its run), the three stages arrive in the same order. The step windows, the lead of induction heads and the time course of the compression differ, and the intermediate median rank does not rebound, although only three checkpoints follow its onset. The scale account of consolidation is therefore a hypothesis within Pythia. We also document an anisotropy correction without which cross-model effective dimensions are incomparable, and a public Pythia checkpoint branch that contains the final weights and mimics a phase transition. This record contains the preprint and a supplementary archive with the per-checkpoint observable records behind every figure and table, the analysis and statistics code, and the figure-generation code. The consolidation test, the trajectory and onset checks, the OLMo comparison and the main figure recompute from the shipped records without a GPU. Code: https://github.com/tao-hpu/jspace-replication.

Zenodo (CERN European Organization for Nuclear Research)
Explainable Artificial Intelligence (XAI)
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon