A Surrogate-Assisted Multi-Phase Ensemble Feature Selection Algorithm With Particle Swarm Optimization in Imbalanced Data

Feature selection based on evolutionary algorithm (EA) is an effective dimension reduction technology. However, existing EAs still constrained by high computational cost and easy-to-local convergence when handling high-dimensional data with imbalanced classes. To this end, a surrogate-assisted multi-phase ensemble feature selection algorithm with particle swarm optimization (SMEFS-PSO) is proposed, which combines the strengths of filter-based feature selection method, surrigate-assisted EA, and the local search strategy. In the first phase of SMEFS-PSO, an ensemble filter feature selection method is adapted to rapidly remove irrelevant and weakly-relevant features. Next, to reduce the identification cost of redundant features, a surrogate-assisted PSO is developed in the second phase. Then, a well-designed problem-specified local search strategy is introduced in the third phase to enhance the local capability. Furthermore, a representative instance selection strategy based on boundary distribution is developed, which construct a surrogate for the whole data. For majority classes, class boundary instances and center instances are selected as representative instances; while for minority classes, oversampling is used to select representative instances. The processing of imbalanced data is effectively integrated into the construction of the surrogate, which not only handles class imbalance problems but also greatly reduces the running cost of the second stage. Finally, the SMEFS-PSO is compared with 9 state-of-art feature selection algorithms on 16 benchmark problems. The experimental results demonstrate that the SMEFS-PSO has superior classification performance with less computing cost for high-dimensional imbalanced feature selection problem.