Condition Number as the Common Currency: A Unified Derivation of Normal-Equation and Gradient-Descent Estimation in Linear Regression under Multicollinearity

Multicollinearity is usually diagnosed with variance inflation factors (VIFs), numerical stability is discussed in terms of the condition number, and the speed of gradient descent is discussed in terms of step sizes and iteration counts. We show that these are three views of a single quantity, the condition number κ of the (standardized) Gram matrix. Starting from elementary calculus (the gradient of the squared-error loss) and matrix algebra (the normal equation), we derive the exact error recursion of gradient descent, its optimal step size and contraction factor ρ* = (κ − 1)/(κ + 1), and two-sided bounds on the iteration count. We then prove the elementary sandwich κ/p² ≤ max_j VIF_j ≤ κ for p standardized predictors, which converts every statement about κ into one about VIFs. Consequences include: the optimal contraction factor is at least the largest absolute pairwise correlation (and equals it for two predictors); the sampling risk of ordinary least squares lies between σ²κ/(np²) and σ²pκ/n; and direct solution via the normal equations loses about log₁₀ κ digits. A flop-count analysis identifies when iterative methods can beat direct ones. Synthetic experiments confirm each bound. The ingredients are classical; the contribution is their assembly into one framework with explicit constants.