An Improved K-Nearest Neighbor Algorithm Based on Entropy Method

针对传统K最近邻(KNN)算法在处理多维度、多量纲数据时，常因特征分布不均衡及量纲差异导致分类性能下降的问题，本文提出了一种基于熵值赋权的改进KNN算法。该方法融合了熵值赋权与标准化欧氏距离的优点，通过引入信息熵来量化各特征的信息量，并依据其重要性构建自适应权重体系，从而在距离计算中对各特征进行差异化处理，减弱了传统距离度量对高权重特征的过度敏感性。实验环节选取UCI公共数据集中的多个数据集进行测试，结果表明改进算法在大部分时候准确率优于传统KNN算法，且在高维数据集上的提升尤为显著。通过参数寻优确定最佳K值后，该算法能够显著提升分类准确率，有效解决传统KNN算法在特征分布不均和量纲差异下的性能不足问题。To address the issue of reduced classification performance in traditional K-Nearest Neighbors (KNN) when dealing with multidimensional and multi-scale data—often caused by imbalanced feature distributions and discrepancies in scales—this paper proposes an improved KNN algorithm based on entropy weighting. This method combines the advantages of entropy weighting and normalized Euclidean distance. By introducing information entropy to quantify the information content of each feature and constructing an adaptive weighting system based on their importance, the approach enables differentiated processing of each feature in the distance calculation, thereby reducing the over-sensitivity of traditional distance measures to high-weight features. Experiments conducted on several datasets from the UCI repository demonstrate that the improved algorithm generally achieves higher accuracy than the traditional KNN, with particularly significant improvements on high-dimensional datasets. After determining the optimal K value through parameter tuning, the algorithm substantially enhances classification accuracy, effectively overcoming the shortcomings of traditional KNN in scenarios with imbalanced feature distributions and scale differences.