Head Start or Process? Decomposing How an Experience-Acquired Abstraction Shapes Later Learning in a Small Language Model

When a network that has acquired an abstraction A learns a second abstraction B faster, does A supply information or change the process by which B is learned? We decompose a gradient learner's function-space update into starting predictions, evidence and tangent kernel, and isolate the kernel channel with an output-matched learner: A's network with a fixed stop-gradient offset, so that it starts B with the naive model's predictions. We apply it to rank-8 adapters on Qwen3-1.7B that learned a hidden two-cue conjunction from outcome-labelled episodes. The compact layer-24 mediator of A's transfer is the answer-letter readout (one adapter tested fully, five on the training domains with fewer candidates). A's raw advantage is a head start plus a network effect that operates only from A's own saturated predictions; from matched starts, A's network gives no detectable mean speed-up. Near-inert replicates of the base learner differ by up to 0.28 in mean held-out balanced accuracy, so single-run comparisons, including our preregistered ones, are dominated by trajectory noise. Against replicated references (post hoc), the mean kernel-channel benefit is below +0.02 at the onset of learning on a task whose rule A does not share, and below +0.09 on a task that extends A, where the scaffold question stays open. Two post hoc observations suggest the method is not blind: all three shuffled-outcome adapters slowed matched-start learning (not significant at the adapter level), and one of four kernel statistics ranked the six adapters' matched-start advantage (Spearman 0.94 on both tasks, uncorrected). The preregistered positive control failed calibration, so the frozen verdict is uninterpretable. Files. head-start-or-process.pdf (the paper); head-start-or-process-source.zip (LaTeX source and the scripts that write every table and figure from the results); preregistrations.zip (the three preregistrations, the SHA-256 record, the deviation log and the frozen forecast); code-and-data.zip (code, the ten A adapters, control adapters, every cell's results, logs and analyses; MANIFEST.json gives the SHA-256 of every file). Code and learned adapters. The code, preregistrations, numeric results and paper source are also at github.com/Somebodyhere101/head-start-or-process. Every learned B adapter (about 2 GB) is attached to its release v1.0 rather than to this record, as b-adapters-part01-of-09.zip to part09, with SHA-256 sums in the release notes. Registered forecast. The F_sign forecast for the four held-out adapters is a separate record, doi:10.5281/zenodo.23432798, published before this preprint. Use of AI assistance. Substantial AI assistance was used throughout this project. The question of whether an experience-acquired abstraction changes the process by which a later abstraction is formed, rather than only supplying information, is the author's. An AI system (Claude, Anthropic), working under the author's direction, audited the earlier studies, reviewed the literature, designed the study, wrote the preregistrations and code, ran the experiments and analyses, and drafted the paper. A separate instance of the same system reviewed the draft; that review led to the post hoc analyses E3 and E4. The preregistrations record that the author delegated the design decisions to the AI system. The author directed the work and is responsible for its content.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-17
DOI
https://doi.org/10.5281/zenodo.23432799
Primary Topic
Domain Adaptation and Few-Shot Learning
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Head Start or Process? Decomposing How an Experience-Acquired Abstraction Shapes Later Learning in a Small Language Model

Sam Ramdan
Zenodo (CERN European Organization for Nuclear Research)
Domain Adaptation and Few-Shot Learning
preprint

Head Start or Process? Decomposing How an Experience-Acquired Abstraction Shapes Later Learning in a Small Language Model

Sam Ramdan
preprint en

Abstract

When a network that has acquired an abstraction A learns a second abstraction B faster, does A supply information or change the process by which B is learned? We decompose a gradient learner's function-space update into starting predictions, evidence and tangent kernel, and isolate the kernel channel with an output-matched learner: A's network with a fixed stop-gradient offset, so that it starts B with the naive model's predictions. We apply it to rank-8 adapters on Qwen3-1.7B that learned a hidden two-cue conjunction from outcome-labelled episodes. The compact layer-24 mediator of A's transfer is the answer-letter readout (one adapter tested fully, five on the training domains with fewer candidates). A's raw advantage is a head start plus a network effect that operates only from A's own saturated predictions; from matched starts, A's network gives no detectable mean speed-up. Near-inert replicates of the base learner differ by up to 0.28 in mean held-out balanced accuracy, so single-run comparisons, including our preregistered ones, are dominated by trajectory noise. Against replicated references (post hoc), the mean kernel-channel benefit is below +0.02 at the onset of learning on a task whose rule A does not share, and below +0.09 on a task that extends A, where the scaffold question stays open. Two post hoc observations suggest the method is not blind: all three shuffled-outcome adapters slowed matched-start learning (not significant at the adapter level), and one of four kernel statistics ranked the six adapters' matched-start advantage (Spearman 0.94 on both tasks, uncorrected). The preregistered positive control failed calibration, so the frozen verdict is uninterpretable. Files. head-start-or-process.pdf (the paper); head-start-or-process-source.zip (LaTeX source and the scripts that write every table and figure from the results); preregistrations.zip (the three preregistrations, the SHA-256 record, the deviation log and the frozen forecast); code-and-data.zip (code, the ten A adapters, control adapters, every cell's results, logs and analyses; MANIFEST.json gives the SHA-256 of every file). Code and learned adapters. The code, preregistrations, numeric results and paper source are also at github.com/Somebodyhere101/head-start-or-process. Every learned B adapter (about 2 GB) is attached to its release v1.0 rather than to this record, as b-adapters-part01-of-09.zip to part09, with SHA-256 sums in the release notes. Registered forecast. The F_sign forecast for the four held-out adapters is a separate record, doi:10.5281/zenodo.23432798, published before this preprint. Use of AI assistance. Substantial AI assistance was used throughout this project. The question of whether an experience-acquired abstraction changes the process by which a later abstraction is formed, rather than only supplying information, is the author's. An AI system (Claude, Anthropic), working under the author's direction, audited the earlier studies, reviewed the literature, designed the study, wrote the preregistrations and code, ran the experiments and analyses, and drafted the paper. A separate instance of the same system reviewed the draft; that review led to the post hoc analyses E3 and E4. The preregistrations record that the author delegated the design decisions to the AI system. The author directed the work and is responsible for its content.

Zenodo (CERN European Organization for Nuclear Research)
Quality education
Domain Adaptation and Few-Shot Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.