Permutation-based multi-objective evolutionary feature selection for high-dimensional data

Abstract Feature selection is a critical step in the analysis of high-dimensional data, where the number of features often vastly exceeds the number of samples. Effective feature selection not only improves model performance and interpretability but also reduces computational costs and mitigates the risk of overfitting. In this context, we propose a novel feature selection method for high-dimensional data, inspired by the well-known permutation feature importance approach. Instead of focusing on individual features, the proposed method evaluates subsets of attributes, offering a more comprehensive analysis of how feature interactions affect model performance. The proposed method employs a multi-objective evolutionary algorithm to search for candidate feature subsets, with the objectives of maximizing the degradation in model performance when the selected features are shuffled, and minimizing the cardinality of the feature subset. The effectiveness of our method has been validated on a set of 27 publicly available high-dimensional datasets for classification and regression tasks, and compared against 13 well-established feature selection methods designed for high-dimensional problems, including the conventional permutation feature importance method. The results demonstrate the ability of our approach in balancing accuracy and computational efficiency, providing a powerful tool for feature selection in complex, high-dimensional datasets.