A Corpus-Domain Overlap Threshold Governs Label-Free Cognometric Transport

Label-free cognometric transport (a linear Procrustes map fit between embedding spaces from an unlabeled corpus, used to score a behavioral axis in a foreign space) is widely useful but unevenly reliable across embedding families. We report a single, bounded empirical regularity: across 17 size-controlled label-free corpora spanning two studies (n=5 + n=12), four OpenAI target models, two foreign embedding spaces, and 75 evaluation prompts, **cross-family transport quality is governed by a measurable threshold in corpus↔domain overlap** (mean-max cosine to the eval prompts in the home space). Below overlap ≈ 0.31, cross-family transported AUC collapses to ~0.69; at and above the threshold, cross-family AUC clears 0.80 and tracks same-family. Same-family transport is essentially insensitive to overlap. A high-resolution 12-level replication recovered the cross-family threshold but **failed the preregistered same-family control criterion** (Spearman −0.41, limit ±0.40); we report this as a real bound on the claim, not a footnote. A separate cross-vendor preregistered test killed any universality reading: the same `mpnet × corpus_2` cell that is worst for OpenAI is worst for Anthropic (min transported 0.617), showing the residual failure mode lives at the corpus/foreign-space boundary, not at the vendor boundary. The claim is therefore narrow: the threshold is a property of the **corpus ↔ foreign-space pairing**, vendor- agnostic in that sense, and only validated same-family. It is not a universal AI-integrity result. Repository: fathom-lab/styxx @ 58a1d98 (PyPI styxx==7.4.1). Bundle contents: paper, figure, raw run JSON (out_corpus_coverage_law.json, out_corpus_coverage_law_fine.json, out_cross_vendor_refusal_transport_confirm.json), scripts (corpus_coverage_law.py, corpus_coverage_law_fine.py, cross_vendor_refusal_transport_confirm.py, plot_threshold_law.py), the related papers establishing the audit chain (corpus-coverage law original + fine replication, cross-vendor stress, cross-vendor preregistration-killed confirmation, refusal-transport stress boundary, styxx status consolidation map, research integrity protocol), and the styxx-on-paper self-audit (threshold-law-self-audit-2026-05-18.md) — the paper scored by the very instruments it documents. Self-audit verdict: 0 cracks requiring revision; all 8 headline numbers match raw JSON within 0.005; integrity protocol rules visibly followed; construct-ceiling firings on the limits/integrity sections are register artifacts predicted by the consolidation map. Honest bounds (also in the paper): the strict preregistered same-family flat-control criterion failed in the high-resolution 12-point replication (Spearman -0.41 vs +/-0.40 limit); an independent cross-vendor preregistration was killed (min Anthropic transported AUC 0.617 below the 0.70 floor). Both are reported in the paper body, not in footnotes. This is a Zenodo methods deposit, not peer-reviewed, no arXiv endorsement claimed, no universality claimed. Lineage: this deposit is methodologically downstream of the Fathom working-paper series (10.5281/zenodo.19609853, 10.5281/zenodo.19502710, 10.5281/zenodo.19468271) and is a supplement to the styxx tool (repo, PyPI 7.4.1, commit 58a1d98). It is not a continuation of the Fathom depth/geometry line; it is a separate, narrower empirical finding about label-free cognometric transport, audited by the same research-integrity protocol. Lineage: this deposit supplements Fathom v23 / styxx v7.2.0 and is methodologically downstream of the Fathom Cognometric series (Every Mind Leaves Vitals, styxx v6.2.0 ref impl, Fathom Cognitive Atlas v0.3). It is a supplement to the styxx tool (repo, PyPI 7.4.1, commit 58a1d98). It is not a continuation of the depth/geometry line; it is a narrower empirical finding about label-free cognometric transport, audited by the same research-integrity protocol.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-05-19
DOI
https://doi.org/10.5281/zenodo.20278945
Primary Topic
Neurobiology of Language and Bilingualism
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

A Corpus-Domain Overlap Threshold Governs Label-Free Cognometric Transport

Alexander Rodabaugh
Zenodo (CERN European Organization for Nuclear Research)
Neurobiology of Language and Bilingualism
article

A Corpus-Domain Overlap Threshold Governs Label-Free Cognometric Transport

Alexander Rodabaugh
article en

Abstract

Label-free cognometric transport (a linear Procrustes map fit between embedding spaces from an unlabeled corpus, used to score a behavioral axis in a foreign space) is widely useful but unevenly reliable across embedding families. We report a single, bounded empirical regularity: across 17 size-controlled label-free corpora spanning two studies (n=5 + n=12), four OpenAI target models, two foreign embedding spaces, and 75 evaluation prompts, **cross-family transport quality is governed by a measurable threshold in corpus↔domain overlap** (mean-max cosine to the eval prompts in the home space). Below overlap ≈ 0.31, cross-family transported AUC collapses to ~0.69; at and above the threshold, cross-family AUC clears 0.80 and tracks same-family. Same-family transport is essentially insensitive to overlap. A high-resolution 12-level replication recovered the cross-family threshold but **failed the preregistered same-family control criterion** (Spearman −0.41, limit ±0.40); we report this as a real bound on the claim, not a footnote. A separate cross-vendor preregistered test killed any universality reading: the same `mpnet × corpus_2` cell that is worst for OpenAI is worst for Anthropic (min transported 0.617), showing the residual failure mode lives at the corpus/foreign-space boundary, not at the vendor boundary. The claim is therefore narrow: the threshold is a property of the **corpus ↔ foreign-space pairing**, vendor- agnostic in that sense, and only validated same-family. It is not a universal AI-integrity result. Repository: fathom-lab/styxx @ 58a1d98 (PyPI styxx==7.4.1). Bundle contents: paper, figure, raw run JSON (out_corpus_coverage_law.json, out_corpus_coverage_law_fine.json, out_cross_vendor_refusal_transport_confirm.json), scripts (corpus_coverage_law.py, corpus_coverage_law_fine.py, cross_vendor_refusal_transport_confirm.py, plot_threshold_law.py), the related papers establishing the audit chain (corpus-coverage law original + fine replication, cross-vendor stress, cross-vendor preregistration-killed confirmation, refusal-transport stress boundary, styxx status consolidation map, research integrity protocol), and the styxx-on-paper self-audit (threshold-law-self-audit-2026-05-18.md) — the paper scored by the very instruments it documents. Self-audit verdict: 0 cracks requiring revision; all 8 headline numbers match raw JSON within 0.005; integrity protocol rules visibly followed; construct-ceiling firings on the limits/integrity sections are register artifacts predicted by the consolidation map. Honest bounds (also in the paper): the strict preregistered same-family flat-control criterion failed in the high-resolution 12-point replication (Spearman -0.41 vs +/-0.40 limit); an independent cross-vendor preregistration was killed (min Anthropic transported AUC 0.617 below the 0.70 floor). Both are reported in the paper body, not in footnotes. This is a Zenodo methods deposit, not peer-reviewed, no arXiv endorsement claimed, no universality claimed. Lineage: this deposit is methodologically downstream of the Fathom working-paper series (10.5281/zenodo.19609853, 10.5281/zenodo.19502710, 10.5281/zenodo.19468271) and is a supplement to the styxx tool (repo, PyPI 7.4.1, commit 58a1d98). It is not a continuation of the Fathom depth/geometry line; it is a separate, narrower empirical finding about label-free cognometric transport, audited by the same research-integrity protocol. Lineage: this deposit supplements Fathom v23 / styxx v7.2.0 and is methodologically downstream of the Fathom Cognometric series (Every Mind Leaves Vitals, styxx v6.2.0 ref impl, Fathom Cognitive Atlas v0.3). It is a supplement to the styxx tool (repo, PyPI 7.4.1, commit 58a1d98). It is not a continuation of the depth/geometry line; it is a narrower empirical finding about label-free cognometric transport, audited by the same research-integrity protocol.

Zenodo (CERN European Organization for Nuclear Research)
Peace, Justice and strong institutions
Openalex Percentile: Top 4%
Neurobiology of Language and Bilingualism
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.