Dropout Is a Data-Dependent Ridge Penalty: Exact Proofs for Linear Regression, Checks on Concrete and Power-Plant Data, and Why the Quadratic Approximation Misses 12% of the Logistic Penalty

Dropout trains a model while randomly deleting inputs. For linear regression with inverted dropout (keep probability p, kept inputs scaled by 1/p), I prove that the expected squared loss equals the ordinary loss plus a ridge penalty with weight (1 - p)/p times the sum of squares of each feature, so the dropout solution is a ridge estimator with feature-specific weights, and on standardised features an ordinary ridge with lambda = n(1 - p)/p. Monte Carlo averages over 40,000 masks match the formula within 0.4% on the UCI Concrete Compressive Strength (n = 1,030) and Combined Cycle Power Plant (n = 9,568) data, and dropout stochastic gradient descent with Polyak averaging lands within 0.045 of the closed-form solution, whose norm is about 10.2. At p = 0.8 dropout shrinks the coefficient vector to 0.56 and 0.67 of the least-squares norm. For logistic regression the penalty is not quadratic. On UCI Spambase (4,601 emails, 57 features) the widely used second-order approximation gives 0.80 to 0.87 of the Monte Carlo penalty for keep probabilities from 0.5 to 0.95. I prove why it does not become exact as p approaches 1: dropping a feature is a jump, not a small perturbation, so the limiting penalty per unit of drop rate is a sum of Bregman divergences of the log-partition function, not of quadratic terms. On Spambase the limiting ratio is 0.880.