Enhancing medical data completeness using an iterative KNN based-Kernelized fuzzy c-means imputation method

Accurate handling of missing values in medical datasets is essential to ensure reliable analysis, accurate diagnoses, and effective treatment planning. Incomplete medical data can significantly compromise the quality of clinical decision-making and hinder the development of intelligent healthcare systems. While numerous missing value imputation (MVI) methods have been proposed in the literature to address this challenge, most suffer from critical limitations such as difficulty in selecting optimal parameters and high sensitivity to noise and outliers. To address these limitations, this study introduces iKNN-KFCM, an iterative K-Nearest Neighbor-based Kernelized Fuzzy C-Means hybrid imputation method, specifically designed to enhance the quality and reliability of imputation in medical data. By synergistically integrating the local learning capability of K-Nearest Neighbors (KNN) with the non-linear clustering power of Kernelized Fuzzy C-Means (KFCM) within an iterative learning process, the proposed method refines missing value estimates by capturing the underlying structure and interrelationships within clinical datasets. Comprehensive experiments were conducted on five real-world medical datasets from the UCI repository with different levels of missing data to evaluate the effectiveness of our proposed iKNN-KFCM imputation method. The performance of this method was compared to eight other state-of-the-art imputation methods (mean, linear interpolation (LI), median, KNN imputation (KNNI), FCM imputation (FKMI), iterative fuzzy clustering (IFC), k-means imputation (KMI), and LI based FCM (LIFCM)) using several criteria such as root mean square error (RMSE), mean absolute error (MAE), mean imputation error (MIE), and mean square error (MSE). Additionally, Friedman’s statistical test was employed to validate the comparative results by ranking each imputation method based on its performance across different evaluation criteria. The experimental results indicated that the iKNN-KFCM method outperformed the existing methods across all evaluation criteria for these datasets, achieving superior performance in 94.29%, 88.57%, and 85.71% of instances for MIE, MAE, and RMSE, respectively, across all missing combinations. These results highlight the effectiveness and robustness of the iKNN-KFCM approach, reinforcing its suitability for improving the reliability of medical data analysis and supporting critical healthcare decision-making processes.