Feature Selection Method Based on Consecutive Forward Selection and Backward Elimination Concepts Using a Weighted Vector

Feature selection is an essential preprocessing task in many disciplines, including Machine Learning (ML) and the Internet of Things (IoT), and it is the most demanding process for data analysis. This process attempts to identify and remove as much irrelevant and redundant information as possible in a controlled manner. Existing algorithms still have limitations in selecting the most informative features maintaining high classification accuracy results. This study proposed a consecutive Forward selection and Backward Elimination algorithm (FBWV) that enhances feature selection by applying the forward selection concept, backward elimination concept, weighted chi-square vector, and custom decision threshold value. The FBWV model framework was optimized through data preprocessing and parameter tuning. The effectiveness of the proposed method was evaluated by comparing it with other state-of-the-art Feature Selection Algorithms (FSA), namely, Rough Set (RS), Weight-Guided (WG) feature selection, and Stability-correlation and Correlation (ScC). The reduced subsets were trained by several classifiers using different measures, including accuracy, F-measure, reduction rate, and AUC. The results revealed that the FBWV effectively reduced the size of the given datasets. It achieved the highest accuracies of 85.28%, 88.33%, 96.26%, 81.36%, 96%, 74.39%, 81.89%, 65.26%, and 98.69% for Austra, Heart Disease, Phishing, Sonar, Iono, SGC, and SpamBase, respectively. The Messidor and Pop-Failure datasets outperform the other FSAs. Moreover, it achieved the highest F-measure and AUC rates of 97.94% each for the Pop-Failure dataset. The FBWV proved the capability of handling different types of datasets and reduced computational complexity, storage, and cost.