Multi-Stage Dimension Reduction for Large-Scale Multi-Objective Feature Selection

Feature selection is a critical step in reducing dimensionality and enhancing the performance of machine learning models, especially when dealing with large-scale datasets. This paper introduces a novel multi-objective feature selection algorithm that aims to maximize classification accuracy while minimizing the ratio of selected features. The approach incorporates a feature space shrinkage mechanism that periodically removes irrelevant features throughout the evolutionary process. By simplifying the search space early in the optimization, the method enhances the algorithm's ability to focus on meaningful solutions, reducing the computational burden and improving classifier performance. The proposed method is evaluated on 12 large-scale datasets with dimensions ranging from 2,400 to nearly 50,000 features. Experimental results demonstrate the method's superiority over the baseline algorithm, showing that unnecessary features can be discarded early without compromising solution quality. This leads to significant improvements in computational efficiency and Pareto-optimal solutions. On average, the proposed method achieves higher hypervolume (HV) values, combining improved classification accuracy with a substantial reduction in the number of selected features.