Greedy Feature Selection Based on Residual Downhill with Sparse Regularization

Feature selection plays an essential role in the field of computer vision. Current research involves the adjustment of feature subsets to minimize the dissimilarity between the feature space and selected feature subset, thus enhancing the quality of the chosen features. However, the optimization process encounters challenges related to extended computation times. To address this challenge, this study introduces an $$L_{2,0}$$ sparse constraint and presents a greedy feature selection approach utilizing a residual downhill strategy to enhance computational efficiency, while not compromising model accuracy. The residual downhill can quickly reduce the loss of the objective function, and the sparse regularization can filter out irrelevant features. This study aims to rigorously evaluate the effectiveness of the model by utilizing various data sets, including six publicly gene datasets related to diseases (e.g., Leukemia etc.), four additional classification challenge datasets (e.g., Arcene, etc.), and five image datasets (e.g., VOC-2007, etc.). KNN and SVM with five-fold cross validation are implemented as classifiers to assess their efficacy. The performance of the model is comprehensively assessed by three evaluation metrics such as classification accuracy. Finally, the results are tested for significance to verify that the improvement is significant based on P-value. Due to the efficacy of this method, it is necessary to further investigate it and explore its applications in other domains.