Performance Evaluation of Diabetes Prediction Model Based on Imbalanced Dataset Using Feature Selection and Hyperparameter Tuning of Classifiers

The performance of a prediction model heavily depends on the distribution of class labels in the dataset. An imbalanced dataset always negatively impacts the performance of the prediction model and necessitates the use of appropriate metrics for evaluation. In this paper, research has been conducted on an imbalanced dataset - PIMA Diabetes Dataset, and the Matthews correlation coefficient has been considered as the evaluation metric for the model. Feature selection methods have also been applied to select only relevant features from the dataset to enhance the performance of the metrics. The features selected by different feature selection methods may or may not be the same; selecting only those features present in all selected lists may reduce the size of the processed dataset to a critical level and potentially decrease the performance of the model. In this paper, a method called weighted index () is proposed, which uses an acceptability parameter (threshold value) to select the most relevant features from feature lists when these lists differ. Furthermore, two additional datasets were created: one in which all missing values were replaced with the mean of the corresponding column, and a second in which samples containing missing values were removed. Feature selection methods were applied to all three datasets, and the weighted index was used to identify common features based on a predefined threshold. Finally, prediction models were built using K‑nearest neighbors, decision tree, random forest, and support vector machine, with the Matthews correlation coefficient as the primary evaluation metric to select the best model and analyze its performance.