HFSCP: A Heuristic Feature Selection Method Based on Clustering and Parabolic Approximation With Multi-Classifier Evaluation

Feature selection plays a crucial role in building efficient and generalizable classification models, particularly in high-dimensional settings where redundant and irrelevant features can impair performance. In this work, we propose the Heuristic Feature Selection method based on Clustering and Parabolic approximation (HFSCP), a novel approach that integrates a global optimization framework with multi-classifier evaluation. HFSCP employs z-score normalization for data preprocessing and encodes each candidate feature subset as a real-valued vector, which is subsequently binarized using a fixed threshold. Specifically, fuzzy c-means clustering is applied to the current population to partition the search space, followed by a parabolic approximation step that refines cluster centers by estimating local optima through quadratic curve fitting via least squares estimation. This combination of clustering and second-order local search enables the algorithm to converge rapidly toward promising regions of the feature space. Uniquely, HFSCP’s objective function is defined as the maximum classification success achieved among three diverse machine learning algorithms, rather than relying on a single classifier. A penalty term proportional to the ratio of selected features is incorporated to promote sparsity and enhance model interpretability. Extensive experiments, using ten-fold cross-validation over 10 independent runs on multiple benchmark datasets, demonstrate that the HFSCP consistently identifies compact feature subsets that yield high classification accuracy across various learning paradigms. The multi-classifier strategy ensures that the selected features are robust and generalizable, addressing the limitations of conventional single-model optimization approaches.