The Base-Point Barrier: Certifying Low-Rank Truncation Costs a Full Evaluation Pass

Deciding what a low-rank truncation costs in accuracy requires evaluating the truncated network, and this paper proves that nothing measured where the network already sits can take the place of that evaluation. Eckart–Young–Mirsky bounds reconstruction error, not accuracy, and the absence of a cheap bound on the accuracy cost has been read as a gap in the literature. It is a property of the problem, and we price the only route that remains. Call a bound base-point cheap when everything it reads is measured where the network already sits: the downstream map's value and derivatives there, to any finite order, which differential calculus calls the jet of the map at that point, together with the spectrum and the data's projections onto the singular directions. We call the resulting obstruction the base-point barrier, and the paper is organized around the dichotomy it produces. Every selection criterion for low-rank truncation either reads only what is measurable where the network already sits, in which case it cannot certify accuracy at any tolerance, or it evaluates the network where the truncation puts it, in which case certifying one retained subset costs a full pass over the evaluation set. There is no third option. Each branch is a theorem. On the first, every valid base-point-cheap bound carries slack at least μ, a quantity counted on the evaluation set and attained by the construction, not merely bounded by it. We exhibit the separating pair twice, once with a C^∞ unit whose jets agree exactly at every order and once with the network's own GELU at a tolerance driven below any prescribed resolution. Of nine published selection criteria, eight lie inside the class, so no reweighting of the singular directions will yield a guarantee about accuracy. On the second, certifying one retained subset to tolerance τ requires at least (A − 2τ)N forward evaluations and N suffice, so the query complexity is Θ(N) and certified pruning over subsets cannot beat exhaustion, while estimation stays cheap at O(τ^−2 log δ^−1) samples. Both are realized inside a public checkpoint. Two networks there have numerically identical base-point jets and bit-identical accuracy of 0.9106, and under the same truncation they separate into 0.9060 and 0.4908. The separation is the value the theorem predicts to ten digits, and the lower figure is the exact accuracy of a constant predictor. The separating network is adversarial by construction, so the theorem bounds what a guarantee can promise and not what a trained network typically does, and the same checkpoint carries the complementary measurement. What follows for practice is a prescription and not a caution. Estimate the accuracy cost and report an interval, since a probabilistic interval costs a number of sampled examples that does not grow with the evaluation set. Do not seek a certificate, since certified pruning over subsets cannot reduce the order of the search. Use the cheap first-order surrogate to order candidates, where it is well calibrated while the downstream is shallow, and not to guarantee that the ordering is safe. And when certainty is required, budget one evaluation per example. The premise itself becomes a diagnostic costing one forward pass. For every direction of every layer it returns whether the construction exists there and a ceiling on the slack it could force, so that a reader can price the barrier on a model of their own. It is measured across 246 weight matrices in five model-and-task combinations, and it orders the expensive outcome at +0.955 over twenty-one layers of three architectures. Pre-print in progress. The file is restricted while the manuscript is under advisor review. Access is granted by share link.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-09-06
DOI
https://doi.org/10.5281/zenodo.22557174
Primary Topic
Stochastic Gradient Optimization Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

The Base-Point Barrier: Certifying Low-Rank Truncation Costs a Full Evaluation Pass

Gustavo Dias Apolinário
Zenodo (CERN European Organization for Nuclear Research)
Stochastic Gradient Optimization Techniques
preprint

The Base-Point Barrier: Certifying Low-Rank Truncation Costs a Full Evaluation Pass

Gustavo Dias Apolinário
preprint en

Abstract

Deciding what a low-rank truncation costs in accuracy requires evaluating the truncated network, and this paper proves that nothing measured where the network already sits can take the place of that evaluation. Eckart–Young–Mirsky bounds reconstruction error, not accuracy, and the absence of a cheap bound on the accuracy cost has been read as a gap in the literature. It is a property of the problem, and we price the only route that remains. Call a bound base-point cheap when everything it reads is measured where the network already sits: the downstream map's value and derivatives there, to any finite order, which differential calculus calls the jet of the map at that point, together with the spectrum and the data's projections onto the singular directions. We call the resulting obstruction the base-point barrier, and the paper is organized around the dichotomy it produces. Every selection criterion for low-rank truncation either reads only what is measurable where the network already sits, in which case it cannot certify accuracy at any tolerance, or it evaluates the network where the truncation puts it, in which case certifying one retained subset costs a full pass over the evaluation set. There is no third option. Each branch is a theorem. On the first, every valid base-point-cheap bound carries slack at least μ, a quantity counted on the evaluation set and attained by the construction, not merely bounded by it. We exhibit the separating pair twice, once with a C^∞ unit whose jets agree exactly at every order and once with the network's own GELU at a tolerance driven below any prescribed resolution. Of nine published selection criteria, eight lie inside the class, so no reweighting of the singular directions will yield a guarantee about accuracy. On the second, certifying one retained subset to tolerance τ requires at least (A − 2τ)N forward evaluations and N suffice, so the query complexity is Θ(N) and certified pruning over subsets cannot beat exhaustion, while estimation stays cheap at O(τ^−2 log δ^−1) samples. Both are realized inside a public checkpoint. Two networks there have numerically identical base-point jets and bit-identical accuracy of 0.9106, and under the same truncation they separate into 0.9060 and 0.4908. The separation is the value the theorem predicts to ten digits, and the lower figure is the exact accuracy of a constant predictor. The separating network is adversarial by construction, so the theorem bounds what a guarantee can promise and not what a trained network typically does, and the same checkpoint carries the complementary measurement. What follows for practice is a prescription and not a caution. Estimate the accuracy cost and report an interval, since a probabilistic interval costs a number of sampled examples that does not grow with the evaluation set. Do not seek a certificate, since certified pruning over subsets cannot reduce the order of the search. Use the cheap first-order surrogate to order candidates, where it is well calibrated while the downstream is shallow, and not to guarantee that the ordering is safe. And when certainty is required, budget one evaluation per example. The premise itself becomes a diagnostic costing one forward pass. For every direction of every layer it returns whether the construction exists there and a ceiling on the slack it could force, so that a reader can price the barrier on a model of their own. It is measured across 246 weight matrices in five model-and-task combinations, and it orders the expensive outcome at +0.955 over twenty-one layers of three architectures. Pre-print in progress. The file is restricted while the manuscript is under advisor review. Access is granted by share link.

Zenodo (CERN European Organization for Nuclear Research)
Universidade Federal de Minas Gerais (BR)
Stochastic Gradient Optimization Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.