A review of LASSO for sparse learning in machine learning

In machine learning, part of the data is used as training data. Applications of high-dimensional data in fields such as biomedical research and text analysis often require prediction and feature selection from limited samples. The Least Absolute Shrinkage and Selection Operator (LASSO) uses an L1 penalty to set some feature coefficients to zero to achieve better feature selection and model prediction. This review examines the sparsity mechanism of LASSO, focusing on the analysis of training procedures in regression and classification, parameter selection, and evaluation method. It also discusses extensions of the method. The literature indicates that LASSO can reduce the number of features while maintaining predictive accuracy when a small set of features carries most of the relevant information. In the use of LASSO, prediction error, the number of features, and selection stability should be considered simultaneously. Depending on the task, adoption of elastic net, stability selection, and nonlinear extensions may be needed.