Feature Stability in Cancer Classification: The Role of Homogeneous Ensembles with ReliefF and Information Gain

Feature selection is pivotal in managing high-dimensional datasets, where the challenge of overfitting and reduced interpretability arises due to the imbalance between the number of features and observations. This study investigates the use of homogeneous ensemble methods to address feature stability in cancer classification, focusing on the ReliefF and Information Gain algorithms within a unified framework. We conducted a comprehensive analysis using stability metrics, such as the Kuncheva Index and assessed classification performance through ROC-AUC scores across various data perturbation levels and ensemble sizes. Our results demonstrate that while both ReliefF and Information Gain benefit from increased data perturbation, ReliefF consistently outperforms Information Gain in terms of stability. Specifically, at higher perturbation levels, ReliefF shows superior stability scores and maintains high accuracy, indicating its robustness in feature selection. This research offers valuable insights into how homogeneous ensemble approaches, particularly using the ReliefF algorithm can enhance feature stability and reliability in predictive modeling. The findings contribute to more robust feature selection methods, with important applications for bioinformatics and other fields dealing with complex, high-dimensional data.