Reading, Not Decomposing: The 24-Cell Polytope Renders Already-Orthogonal Semantic Features in Real LLM Sentence Embeddings

The Watabe-Claude Method (WCM) discretizes 4-dimensional point clouds with the 24-cell regular polytope(the F4 root system). We ask whether this geometry can read out, or decompose, the multiple semanticfeatures that a large language model superposes in a single embedding vector. Using genuine sentenceembeddings from Llama-3.2-3B-Instruct (n_embd=3072) obtained locally via ollama, we find: (1) three topicpoles (AI / arts / society) map to distinct 24-cell vertices at purity 1.000, 11σ above a label-shuffle null,reproducing a 2026 BERT result on a different model, corpus and method; (2) when topic and emotionalsentiment are BOTH varied within each sentence, the 24-cell's coordinate axes divide the labor cleanly — thex0-x1 plane carries topic (1.000 CV accuracy, sentiment at chance), the x2 axis carries sentiment (0.956, topicat chance), and the sign of x2 matches sentiment on 100% of sign-bearing sentences, 6σ above asentiment-shuffle null; (3) a third, weaker feature (tense) does not surface on the raw top-4 principal axes butcan be aligned onto x3 when its cues are distributed across the sentence, raising its raw-projectionrecoverability from chance (0.500) to 0.855. Crucially, (4) on an Anthropic-faithful synthetic model of TRUEsuperposition (8 sparse features non-orthogonally compressed into 4 dimensions), 24-cell readout does NOTexceed the raw compressed representation (0.687 vs 0.711) — because projection onto the 24 vertices is arank-4 linear map that adds no information, and its only nonlinearity (vertex argmax) discards overlap. Weconclude that the 24-cell is a readout / rendering tool for features a model already representsnear-orthogonally, not a decomposition tool for genuine superposition. This delimitation is the paper's centralcontribution. ※ The License for codes is AGPL 3.0-or-later

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-08-14
DOI
https://doi.org/10.5281/zenodo.21937160
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Reading, Not Decomposing: The 24-Cell Polytope Renders Already-Orthogonal Semantic Features in Real LLM Sentence Embeddings

Masanori Watabe, Claude Opus 4.8 Kurado
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

Reading, Not Decomposing: The 24-Cell Polytope Renders Already-Orthogonal Semantic Features in Real LLM Sentence Embeddings

Masanori Watabe, Claude Opus 4.8 Kurado
preprint en

Abstract

The Watabe-Claude Method (WCM) discretizes 4-dimensional point clouds with the 24-cell regular polytope(the F4 root system). We ask whether this geometry can read out, or decompose, the multiple semanticfeatures that a large language model superposes in a single embedding vector. Using genuine sentenceembeddings from Llama-3.2-3B-Instruct (n_embd=3072) obtained locally via ollama, we find: (1) three topicpoles (AI / arts / society) map to distinct 24-cell vertices at purity 1.000, 11σ above a label-shuffle null,reproducing a 2026 BERT result on a different model, corpus and method; (2) when topic and emotionalsentiment are BOTH varied within each sentence, the 24-cell's coordinate axes divide the labor cleanly — thex0-x1 plane carries topic (1.000 CV accuracy, sentiment at chance), the x2 axis carries sentiment (0.956, topicat chance), and the sign of x2 matches sentiment on 100% of sign-bearing sentences, 6σ above asentiment-shuffle null; (3) a third, weaker feature (tense) does not surface on the raw top-4 principal axes butcan be aligned onto x3 when its cues are distributed across the sentence, raising its raw-projectionrecoverability from chance (0.500) to 0.855. Crucially, (4) on an Anthropic-faithful synthetic model of TRUEsuperposition (8 sparse features non-orthogonally compressed into 4 dimensions), 24-cell readout does NOTexceed the raw compressed representation (0.687 vs 0.711) — because projection onto the 24 vertices is arank-4 linear map that adds no information, and its only nonlinearity (vertex argmax) discards overlap. Weconclude that the 24-cell is a readout / rendering tool for features a model already representsnear-orthogonally, not a decomposition tool for genuine superposition. This delimitation is the paper's centralcontribution. ※ The License for codes is AGPL 3.0-or-later

Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.