The Influence of Regularization Intensity on the Bias Variance of Linear Regression

This study proposes an L2 regularization-based framework for optimizing linear regression models' generalization ability. Through comparative analysis of ordinary least squares (OLS) and ridge-like models on synthetic data, we investigate regularization's role in bias-variance trade-off. The experimental protocol involves: (1) generating linear data (y = 3X + 5 + ϵ) with Gaussian noise (σ = 2), (2) estimating OLS parameters via normal equations, and (3) implementing gradient descent with regularization terms (λ ∈ {0.0,0.01,0.1,1.0}), using 2λθ_j for weight correction. Results show the λ = 0.1 model achieves optimal MSE(Mean Squared Error) performance (MSE = 4.21), 15.3% better than OLS (MSE = 4.97), with parameters (intercept = 5.12,coefficient = 2.98) closer to true values. Visual analysis confirms the regularized model's superior robustness in feature distribution edges, contrasting with OLS's overfitting tendency. The proposed grid search and gradient correction methods provide an interpretable framework for lightweight model optimization, extendable to elastic networks and deep neural networks in high-dimensional scenarios.