Beyond Shared Geometry: Predictive Recovery, Exact Actions, and Registration Preference in Language-Model Representations
Predictive maps between language-model representations need not identify the transformation structure those representations support. We study finite-group actions on visible input states with coordinates shared across contexts. A spectral budget retaining complete real irreducible components bounds the minimum prediction risk over exact orthogonal actions of a fixed real dimension. Under a common second-moment-whitened loss, its upper endpoint is at most twice its lower endpoint. An analytical example shows that the ordinary orbit-Gram spectrum does not determine this exact-action risk. On fixed banks from six frozen Llama and Qwen models, we separate independent orthogonality cost from bounds on the further cost of common group laws. These finite-bank class comparisons are distinct from held-out prediction. Training-constructed exact actions predict held-out representations less accurately than training-fitted independent least squares. Spectral registration preference depends on retained dimension and state-invariant components. At dimension 16, within-template centering with the original whitening changes the prescribed image's unique leads from 7/24 to 15/24 weekday (C₇) conditions and from 16/24 to 0/24 corner (D₄) conditions. Rewhitening raises the weekday count to 18/24 while corners remain at 0/24. On separately authored contexts, difference fitting lowers mean weekday error and raises mean corner error relative to coordinate fitting; wording, validation selection and, for corners, action weighting also differ from the earlier protocol. These comparisons distinguish predictive transport, fixed-bank exact-action compatibility and spectral preference from evidence of native computation.
Authors
- Pengcheng Yang (ORCID: https://orcid.org/0009-0008-1550-9113)
Institutions
- Purdue University West Lafayette (US)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-15
- DOI
- https://doi.org/10.5281/zenodo.24238470
- Primary Topic
- Topic Modeling
- Type
- preprint