Empirical Studies of Multimodality in Feature Selection: Generalized Crowding for Genetic Algorithms

Feature selection methods are important for machine learning (ML), as they can make learned models more accurate, easier to understand, and reduce their size and resource consumption. Interestingly, these methods may also induce multimodal fitness landscapes that are non-trivial to find optima in. In this work, we consider multimodal fitness landscapes induced by feature selection for ML by means of random forests. Among the genetic algorithms developed to handle multimodality, we study the crowding approach, specifically generalized crowding. Empirically, we consider how varying the scaling factor of generalized crowding influences the number of global and local optima found when performing feature selection for random forests for different datasets. While the real-world feature selection problems studied are modest in size, we hope that this research will enable improved understanding of and performance for larger feature selection problems in the future.