Exact Finite-Sample Risk of Minimum-Norm Interpolation: Proofs of the Double-Descent Curve for Isotropic Gaussian Least Squares, with Simulation Checks and the Ridge Comparison

For linear regression with isotropic Gaussian features, n samples, p parameters, signal energy r² and noise variance σ², I prove the exact finite-sample excess risk of the least-squares or minimum-norm interpolating estimator. It is σ²p/(n−p−1) for p ≤ n−2 and r²(1−n/p) + σ²n/(p−n−1) for p ≥ n+2. The proofs use the mean of the inverse Wishart distribution and the rotational invariance of random projections. The formulas reproduce the Hastie–Montanari–Rosset–Tibshirani asymptotics as n, p → ∞ and explain the risk spike at p ≈ n. Simulations with n = 100 and r²/σ² = 4 (400 replications per point) agree with the formulas to within two standard errors wherever |p − n| ≥ 5. Near p = n they become unstable, and I show this is predicted: the second moment of the risk is infinite when |p − n| ≤ 3. At p/n = 2 the min-norm risk is 0.75. That is below the null risk r² = 1 but about 17% above ridge regression at the Bayes-optimal penalty (0.65). In this model, "benign" overparameterisation is a statement about the min-norm estimator, not about the value of interpolation.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-18
DOI
https://doi.org/10.5281/zenodo.23450082
Primary Topic
Statistical Methods and Inference
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Exact Finite-Sample Risk of Minimum-Norm Interpolation: Proofs of the Double-Descent Curve for Isotropic Gaussian Least Squares, with Simulation Checks and the Ridge Comparison

Manoj Kumar Aguru
Zenodo (CERN European Organization for Nuclear Research)
Statistical Methods and Inference
preprint

Exact Finite-Sample Risk of Minimum-Norm Interpolation: Proofs of the Double-Descent Curve for Isotropic Gaussian Least Squares, with Simulation Checks and the Ridge Comparison

Manoj Kumar Aguru
preprint en

Abstract

For linear regression with isotropic Gaussian features, n samples, p parameters, signal energy r² and noise variance σ², I prove the exact finite-sample excess risk of the least-squares or minimum-norm interpolating estimator. It is σ²p/(n−p−1) for p ≤ n−2 and r²(1−n/p) + σ²n/(p−n−1) for p ≥ n+2. The proofs use the mean of the inverse Wishart distribution and the rotational invariance of random projections. The formulas reproduce the Hastie–Montanari–Rosset–Tibshirani asymptotics as n, p → ∞ and explain the risk spike at p ≈ n. Simulations with n = 100 and r²/σ² = 4 (400 replications per point) agree with the formulas to within two standard errors wherever |p − n| ≥ 5. Near p = n they become unstable, and I show this is predicted: the second moment of the risk is infinite when |p − n| ≤ 3. At p/n = 2 the min-norm risk is 0.75. That is below the null risk r² = 1 but about 17% above ridge regression at the Bayes-optimal penalty (0.65). In this model, "benign" overparameterisation is a statement about the min-norm estimator, not about the value of interpolation.

Zenodo (CERN European Organization for Nuclear Research)
Statistical Methods and Inference
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.