Innovative Initialization Scheme for Multi-Objective Feature Selection in Continuous Search Spaces

Feature selection is a demanding and costly endeavor within the realms of machine learning and data mining, targeting the elimination of irrelevant and redundant features. This endeavor significantly bolsters classification accuracy or other post processing components, such as search or clustering. In this context, the feature selection can be as a single-objective optimization task, with the primary objective of maximizing classification accuracy or multi-objective one while minimizing the number of selected features as well which can be tackled using population-based metaheuristic algorithms. Considering that feature selection (FS) inherently poses a binary problem, given that most metaheuristic algorithms operate in continuous domains (such as Real-coded GA, DE, PSO, and CMA-ES), transitioning them to binary search spaces necessitates substantial operator modifications. One of the crucial steps of the population-based algorithms is initialization of the population which significantly impacts both the convergence speed and the quality of the final solution. However, in most cases, especially with continuous algorithms, random initialization emerges as the predominant method for generating candidate solutions (initial population) which lacks diversity in the large-scale feature selection search spaces. In this paper, two novel population initialization methods within the continuous search space are introduced, followed by a comparative analysis against the traditional random initialization method. Experimental results conducted on ten datasets with 300 to 11,000 features demonstrate the effectiveness of population-level uniform initialization, surpassing the widely recognized individual-level uniform initialization method. The experiments are conducted using Differential Evolution as the single-objective algorithm and Generalized Differential Evolution3 as a multi-objective algorithm as our case studies. This study demonstrates the crucial role of initialization in the population-based optimization algorithms when they tackle binary problems.