Sparse Learning-Based Feature Selection in Classification: A Multi-Objective Perspective

Sparse learning-based feature selection is an emerging topic, acclaimed for its potential in delivering promising performance and interpretability. Nevertheless, the task of determining a suitable regularization parameter to strike a balance between the loss function and regularization is a challenging endeavor, where existing methods encounter great difficulties. Moreover, the ranking mechanism in most sparse learning-based feature selection methods requires a predefined number of selected features, which is usually dataset-dependent and not known in advance. It is of great importance to automatically balance the loss function and sparse regularization and determine the appropriate number of selected features. To this end, this paper proposes formulating the sparse learning-based feature selection problem as a bi-objective optimization problem, which takes the loss term and the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{2,0}$</tex-math></inline-formula>-norm regularization as two objectives, to automatically identify the optimal number of selected features and obtain a set of trade-off solutions between the loss term and the number of selected features. To solve such a non-convex problem, a novel solution representation, an initialization strategy, and an environmental selection operator are proposed. Compared with seven feature selection methods, extensive experiments on 16 practical classification datasets demonstrate that the proposed method attains highly competitive classification accuracy with a small number of selected features, and the features selected by the proposed method have low redundancy.