A Method for Improving KNN Model Performance Based on Surrogate Model Feature Selection

The K-Nearest Neighbors (KNN) algorithm is a simple yet effective classification method, but its performance often degrades with high-dimensional data due to interference from irrelevant or redundant features. Traditional KNN treats all features as equally contributing to classification by default, lacking the ability to adaptively characterize feature importance, leading to distance metric bias and reduced classification accuracy. To address this issue, this paper proposes a feature importance evaluation method based on surrogate models to optimize KNN's classification performance. This method utilizes lightweight surrogate models (such as decision trees or linear classifiers) to automatically learn the importance weights of features, and subsequently selects the most discriminative feature subset, thereby enhancing the robustness and computational efficiency of the KNN model. Experimental results show that compared to traditional KNN and mainstream feature weighting methods, the proposed method significantly improves classification accuracy across multiple public datasets while reducing computational overhead. Additionally, this method exhibits good interpretability, effectively identifying key discriminative features. This study provides a new approach for improving the KNN algorithm, particularly suitable for classification tasks involving high-dimensional and imbalanced data.