Exact Finite-Sample Risk of Minimum-Norm Interpolation: Proofs of the Double-Descent Curve for Isotropic Gaussian Least Squares, with Simulation Checks and the Ridge Comparison
For linear regression with isotropic Gaussian features, n samples, p parameters, signal energy r² and noise variance σ², I prove the exact finite-sample excess risk of the least-squares or minimum-norm interpolating estimator. It is σ²p/(n−p−1) for p ≤ n−2 and r²(1−n/p) + σ²n/(p−n−1) for p ≥ n+2. The proofs use the mean of the inverse Wishart distribution and the rotational invariance of random projections. The formulas reproduce the Hastie–Montanari–Rosset–Tibshirani asymptotics as n, p → ∞ and explain the risk spike at p ≈ n. Simulations with n = 100 and r²/σ² = 4 (400 replications per point) agree with the formulas to within two standard errors wherever |p − n| ≥ 5. Near p = n they become unstable, and I show this is predicted: the second moment of the risk is infinite when |p − n| ≤ 3. At p/n = 2 the min-norm risk is 0.75. That is below the null risk r² = 1 but about 17% above ridge regression at the Bayes-optimal penalty (0.65). In this model, "benign" overparameterisation is a statement about the min-norm estimator, not about the value of interpolation.
Authors
- Manoj Kumar Aguru
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-18
- DOI
- https://doi.org/10.5281/zenodo.23450082
- Primary Topic
- Statistical Methods and Inference
- Type
- preprint