Assessing the effectiveness of feature selection methods in machine learning using statistical metrics

Machine learning-based feature selection algorithms evaluate feature relevance using specialized techniques. In many datasets, numerous attributes are often irrelevant to classification tasks. This research underscores the benefits of dimensionality reduction, particularly the elimination of unnecessary attributes, to enhance classification speed while preserving accuracy with essential features. We utilize widely used methods such as Random Forest (RF) and Permutation Feature Importance (PFI) to identify significant features from well-known datasets provided by the University of California, Irvine (UCI), Kaggle, and GitHub. Additionally, statistical techniques— including statistical hypothesis test— based measures such as the Kolmogorov-Smirnov (KS) statistic and the t-test, as well as statistical distance metrics such as Mahalanobis Distance (MD), Bhattacharyya Distance (BD), and Jeffries-Matusita Distance (JMD)— are applied to assess feature relevance effectively. These statistical methods validate the importance of the identified features, establishing strong relationships between critical features and statistical metrics. This research aims to provide valuable insights for enhancing the efficiency of disease detection by leveraging strategic feature selection methods and statistical validation techniques.