Machine Learning Ensemble Techniques for Prediction of Diabetes

This study aims to explore hybrid machine learning techniques for the predictive purpose of diabetes disease. The primary objective is to analyze existing supervised machine learning algorithms like K-Nearest Neighbours (KNN), Naive Bayes (NB), Decision Tree (DT) and so on, that are used for predictive purposes by carrying out extensive research. Through this study, several hybrid machine learning prediction classifiers using supervised techniques and ensemble methods will be studied and evaluated based on their performance matrix of accuracy, precision, recall and F1-score as well. This is to discover which predictive technique is the most suitable for predicting diabetes as well as to evaluate by how much ensemble methods can improve diabetes prediction accuracy. This study utilizes feature selection with Recursive Feature Elimination with Cross Validation (RFECV) to determine the most important features of the dataset, as well as K-Fold Cross Validation to evaluate the performance of all the machine learning techniques used by ensuring that different portions of the dataset is used for training and testing. The results of this study reveal that boosting, by Gradient Boosting (GB), is the best performing ensemble technique at 85.904% accuracy, 70.921 %) precision, 56.840% recall and 58.439% F1-score. For bagging ensemble techniques, Random Forest (RF) had the highest performance at 85.903% accuracy, 71.019% precision, 56.486% recall and 57.939% F1-score. On the other hand, for stacking ensemble techniques, Decision Tree (DT) + Support Vector Machine (SVM) proved to be the highest performing combination with an accuracy of 85.870%, precision of 70.585%, recall of 57.524% and F1-score of 59.358%. For the base supervised techniques itself, K-Nearest Neighbours (KNN) performed almost as well as GB, with the same accuracy as GB (85.904 %), and got a higher precision of 71.050% but had a recall and F1-score that's marginally lower at 56.380% and 57.789% respectively. These findings are significant because, when compared to other existing studies that used the same dataset (Behavioral Risk Factor Surveillance System – 2015 Survey), the performance accuracy of the hybrid machine learning techniques in this study was higher, providing new insights on possible suitable processes taken in this study that can produce better results with the dataset used. To conclude, based on these outcomes we can say that hybrid machine learning techniques generally perform better when compared to supervised machine learning techniques and have the potential to contribute well towards future implementations in the diagnostic medicine field, paving new possibilities to revolutionize traditional diagnostic practices in order to help with the early intervention of diabetes by predictive methods.