Penalty free variable selection for high-dimensional linear models

Variable selection is important in high-dimensional data analysis because it can select a few important explanatory variables from many candidates to better interpret the response. In recent decades, well-known variable selection methods for high-dimensional linear models are mostly developed by the penalized least squares. It is believed that penalties must be used to select the variables. This research points out that variables can also be selected by the combination of the ordinary least squares with the generalized information criterion, leading to a penalty-free variable selection method for high-dimensional linear models. The proposed method has two steps. The first uses the ordinary least squares only. The goal is to select variables based on a given model size. The second step uses the generalized information criterion. The goal is to estimate the model size. The proposed penalty-free variable selection method is derived from the combination of the two steps. Theoretical and numerical studies show that the proposed method’s computational complexity is linear and it outperforms the penalized least squares in general.