The Illusion of Scale: Approximation, Identifiability, and the Entropy Tax

Predictive error and structural error are quantities on different spaces, joined only through the inverse of an observation operator. This note makes that joint explicit. It defines a query-relative inverse modulus and shows that predictive convergence transfers to structural convergence, uniformly over an admissible class, exactly when that modulus vanishes in the limit as the observational tolerance tends to zero. Three regimes follow: strict non-identifiability, unstable inversion, and practical non-identifiability. They separate the cases in which additional scale cannot help from those in which it can. A two-point argument shows that no estimator, and therefore no quantity of computation, distinguishes hypotheses with identical observational laws; a compactness argument shows that, under the stated conditions, restricting the admissible class can convert query-level uniqueness into uniform stability over that class. The resulting picture is an accounting identity: identification is paid for in data, interventions, assumptions, or proofs, and computation searches only whatever those payments leave behind. None of the individual mathematical facts is new. The contribution is a single object that makes the scaling claim precise, and that states the conditions under which the claim is false. Contents: illusion_of_scale_note.pdf is the full note (10 pages, version 1.0). visual_abstract.pdf is a one-page visual abstract of the same material. The LaTeX sources for both documents are included.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-08-24
DOI
https://doi.org/10.5281/zenodo.22082261
Primary Topic
Ethics and Social Impacts of AI
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

The Illusion of Scale: Approximation, Identifiability, and the Entropy Tax

Suvajit Das
Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
preprint

The Illusion of Scale: Approximation, Identifiability, and the Entropy Tax

Suvajit Das
preprint en

Abstract

Predictive error and structural error are quantities on different spaces, joined only through the inverse of an observation operator. This note makes that joint explicit. It defines a query-relative inverse modulus and shows that predictive convergence transfers to structural convergence, uniformly over an admissible class, exactly when that modulus vanishes in the limit as the observational tolerance tends to zero. Three regimes follow: strict non-identifiability, unstable inversion, and practical non-identifiability. They separate the cases in which additional scale cannot help from those in which it can. A two-point argument shows that no estimator, and therefore no quantity of computation, distinguishes hypotheses with identical observational laws; a compactness argument shows that, under the stated conditions, restricting the admissible class can convert query-level uniqueness into uniform stability over that class. The resulting picture is an accounting identity: identification is paid for in data, interventions, assumptions, or proofs, and computation searches only whatever those payments leave behind. None of the individual mathematical facts is new. The contribution is a single object that makes the scaling claim precise, and that states the conditions under which the claim is false. Contents: illusion_of_scale_note.pdf is the full note (10 pages, version 1.0). visual_abstract.pdf is a one-page visual abstract of the same material. The LaTeX sources for both documents are included.

Zenodo (CERN European Organization for Nuclear Research)
Independent Dance (GB)
Peace, Justice and strong institutions
Ethics and Social Impacts of AI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.