Backward stepwise ridge regression and the Lasso: how do they do as finders of true regressors and their coefficients?

For regressor subset selection, three algorithms are introduced. Two use ridge regression (RR), with Lawless and Wang’s parameter (kLW). The third (RLasso) is Lasso, but with kLW or Hoerl, Kennard, and Baldwin’s tuning parameter (kHKB). Backward stepwise elimination (BSE) first fits RR to the full data. Then, the most insignificant regressor is dropped; and RR is fit to the remaining regressors. This repeats until no insignificant regressor remains. Backward stepwise group elimination (BSGE) RR works exactly like BSE RR except that <i>all</i> insignificant regressors are dropped each time. It is practical to do signless coefficient t-testing with a p-value of 15%. Lasso estimation uses Mallows Cp criterion-based parameter (kCp). Accuracy in predicting and finding coefficients are assessed by simulation because the assessment is mathematically intractable. BSE RR, BSGE RR and RLasso surpass Lasso. BSE RR predicts better than BSGE RR. When kCp exceeds kLW or kHKB, RLasso prediction accuracy increases even more. RLasso with kHKB predicts better than RLasso with kLW. Napier’s profound idea is invoked, by introducing computer-simulated “regression tables” to locate input-data regions for maximizing the chances for model discovery. Championing the oracular property in finite samples is cautioned. RR uses “elementary” arithmetic to surpass Lasso, advancing human-machine interface simplicity.