Invariant Ordinal Cores: How Transformers Represent, Compare, and Route Continuous Physical Scales

Algorithmic Core Extraction (ACE) recently demonstrated that transformers solve discrete tasks—Markov state transitions, modular addition, subject–verb agreement, and indirect object identification—through compact, causally necessary and sufficient subspaces shared across seeds and architectures. Yet much of real-world reasoning operates not on discrete symbols, but on continuous physical and semantic orderings: ranking planets by diameter, animals by mass, chemical elements by atomic number, or countries by land area. Here we extend ACE to continuous ordinal ranking across pretrained language models (Qwen2.5-1.5B, Qwen3-1.7B, and Qwen3-0.6B), introducing paired-swap ACE to cancel prompt-template variance and strictly causal pre-context entity extraction to isolate intrinsic entity representations before a comparison target enters the causal attention window. Across models and domains, ordinal comparison decomposes into a shared three-stage algorithmic core pipeline: (1) an intrinsic midlayer entity magnitude register (layers 9–12, 32–43% depth) where a single 1D direction linearly encodes physical log-scale—recovering planetary log10 (diameter) (𝑟 = 0.923–0.994) and atomic number log10 (𝑍) across 20 chemical elements (𝑟 = 0.942–0.966, unseen elements 𝑟 = 0.977)—and permits single-scalar counterfactual transplants; (2) a late-layer lexicon-invariant quantitative comparator bottleneck (layers 19–21, 68–75% depth), where a 1D core extracted from only six planets transfers zeroshot across unseen planets, animals (heavier/lighter), and numbers (higher/lower, Flip AUC = 0.021, 𝜏 = −1.00); and (3) lexical readout divergence (layers 22–28), where vocabulary-specific heads fan out while propositional Yes/No remains orthogonal. Finally, we turn this mechanistic pipeline into CoreRank, a zero-fine-tuning 𝑂(𝑁) pointwise ranking readout from the Stage-1 entity core. Across 3 models and 3 strictly held-out real-world domains under clean and adversarial distractor prompts, CoreRank raises unseen-entity accuracy from 52.7% to 72.0% overall (57.6% to 81.8% on quantitative scales; AUC 0.753 → 0.903), unlocks Qwen3-0.6B from chance (50.0%, AUC 0.351) to 80.0% (0.895 AUC) on unseen elements, cuts prompt-distractor verdict flips 5.7× (25.5% → 4.5%), and mathematically eliminates position bias (67.3% → 0.0%) and intransitivity cycles (54.4% → 0.0%).

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-21
DOI
https://doi.org/10.5281/zenodo.24413357
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Invariant Ordinal Cores: How Transformers Represent, Compare, and Route Continuous Physical Scales

Marcio Diaz
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
article

Invariant Ordinal Cores: How Transformers Represent, Compare, and Route Continuous Physical Scales

Marcio Diaz
article en

Abstract

Algorithmic Core Extraction (ACE) recently demonstrated that transformers solve discrete tasks—Markov state transitions, modular addition, subject–verb agreement, and indirect object identification—through compact, causally necessary and sufficient subspaces shared across seeds and architectures. Yet much of real-world reasoning operates not on discrete symbols, but on continuous physical and semantic orderings: ranking planets by diameter, animals by mass, chemical elements by atomic number, or countries by land area. Here we extend ACE to continuous ordinal ranking across pretrained language models (Qwen2.5-1.5B, Qwen3-1.7B, and Qwen3-0.6B), introducing paired-swap ACE to cancel prompt-template variance and strictly causal pre-context entity extraction to isolate intrinsic entity representations before a comparison target enters the causal attention window. Across models and domains, ordinal comparison decomposes into a shared three-stage algorithmic core pipeline: (1) an intrinsic midlayer entity magnitude register (layers 9–12, 32–43% depth) where a single 1D direction linearly encodes physical log-scale—recovering planetary log10 (diameter) (𝑟 = 0.923–0.994) and atomic number log10 (𝑍) across 20 chemical elements (𝑟 = 0.942–0.966, unseen elements 𝑟 = 0.977)—and permits single-scalar counterfactual transplants; (2) a late-layer lexicon-invariant quantitative comparator bottleneck (layers 19–21, 68–75% depth), where a 1D core extracted from only six planets transfers zeroshot across unseen planets, animals (heavier/lighter), and numbers (higher/lower, Flip AUC = 0.021, 𝜏 = −1.00); and (3) lexical readout divergence (layers 22–28), where vocabulary-specific heads fan out while propositional Yes/No remains orthogonal. Finally, we turn this mechanistic pipeline into CoreRank, a zero-fine-tuning 𝑂(𝑁) pointwise ranking readout from the Stage-1 entity core. Across 3 models and 3 strictly held-out real-world domains under clean and adversarial distractor prompts, CoreRank raises unseen-entity accuracy from 52.7% to 72.0% overall (57.6% to 81.8% on quantitative scales; AUC 0.753 → 0.903), unlocks Qwen3-0.6B from chance (50.0%, AUC 0.351) to 80.0% (0.895 AUC) on unseen elements, cuts prompt-distractor verdict flips 5.7× (25.5% → 4.5%), and mathematically eliminates position bias (67.3% → 0.0%) and intransitivity cycles (54.4% → 0.0%).

Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
Invariant Ordinal Cores: How Transformers Represent, Compare, and Route Continuous Physical Scales — Marcio Diaz · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS