A Comparison of Penalized Regression Methods on Model Estimation and Variable Selection: A Simulation Study

Objective: The aim of this study is to determine which variables to include in the model and to examine how successful the model estimation is with the Least Angle Regression (LAR), Least Absolute Shrinkage Selection Operator (LASSO) and Elastic Net (EN) regression methods, which are alternatives to unbiased methods. Material and Methods: In this study, variables that LAR, LASSO and EN regression methods, which are among the biased methods, take in model selection and their model prediction success are compared. For this purpose, data sets were generated in different scenarios in R program. The results obtained after data sets produced at the end of the simulation with standard normal distribution, sample sizes n=50, n=100, n=200, number of independent variables p=16, p=18, p=20 and correlation coefficients r=0.10; r=0.60; r=0.90 were recorded. Results: Model predictions of the methods were recorded in the study results. While the model predictions of the LAR and LASSO methods were close to each other, the EN regression method differed in model prediction. When analyzed in terms of Mean Square Error (MSE) and Coefficients of Determination, close values were observed. Conclusion: While model prediction success is high in data sets with low sample size, model prediction success decreases and MSE values increase when the sample size increases gradually. For this reason, it has been observed that these methods are more useful and provide better model prediction in cases where there is a multicollinearity problem in the data sets and in scenarios where the sample size is small.