Shapley-Based Data Valuation with Mutual Information: A Key to Modified K-Nearest Neighbors

The K-Nearest Neighbors (KNN) algorithm is widely used for classification and regression; however, it suffers from limitations, including the equal treatment of all samples. We propose Information-Modified KNN (IM-KNN), a novel approach that leverages Mutual Information (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathcal{I}$</tex>) and Shapley values to assign weighted values to neighbors, thereby bridging the gap in treating all samples with the same value and weight. On average, IM-KNN improves the accuracy, precision, and recall of traditional KNN by <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$16.80 \%, 17.08 \%$</tex>, and 16.98 %, respectively, across 12 benchmark datasets. Experiments on four large-scale datasets further highlight IMKNN's robustness to noise, imbalanced data, and skewed distributions.