Leakage-Aware and Reproducible Comparison of Eight Machine Learning Classifiers for the Diagnostic Classification of Heart Disease

Machine-learning studies of heart disease frequently report near-perfect accuracy on the Cleveland benchmark; however, direct comparison of results is hindered by inconsistencies in preprocessing, data partitioning and evaluation metrics across studies. We present a leakage-aware comparison of eight classifiers (logistic regression, naive Bayes, support vector machine, k-nearest neighbours, decision tree, random forest, gradient boosting and a multilayer perceptron) for the diagnostic classification of angiographically defined coronary artery disease. A widely circulated 1,025-record file labelled as Cleveland data contains more records than the combined total of patients in the four UCI heart-disease cohorts (920) and therefore necessarily includes repeated records. All analyses were therefore repeated on the original Cleveland cohort (n = 303, 139 with disease) with ten repetitions of stratified 10-fold cross-validation, and with imputation, scaling, encoding and outlier handling fitted inside each training fold. ROC–AUC was computed from predicted probabilities, and calibration, tuning, feature availability, sex-specific performance and external validation on the Hungarian cohort (n = 294) were examined. Logistic regression had the highest mean AUC (0.911 ± 0.051; accuracy 0.842 ± 0.061), but after Holm correction it could not be separated from random forest, SVM, the multilayer perceptron, knearest neighbours or naive Bayes; only the decision tree and gradient boosting were inferior. Adding 722 repeated records to the unique patients raised random forest accuracy from 0.820 to 0.992, which reproduces the near-perfect accuracy (0.997) obtained with the 1,025-record file. Without ca and thal, the AUC of logistic regression was 0.876, and a resting-variable set gave 0.842. External AUC values ranged from 0.867 to 0.886 for six of the eight classifiers. The results pertain to the diagnostic classification of prevalent disease in referred patients rather than prospective risk prediction, and establish a transparent methodological baseline rather than a model intended for clinical deployment.

Authors

Publication Details

Journal
Mathematics Open
Published
2026-11-13
DOI
https://doi.org/10.1142/s2811007226500318
Primary Topic
Artificial Intelligence in Healthcare
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Leakage-Aware and Reproducible Comparison of Eight Machine Learning Classifiers for the Diagnostic Classification of Heart Disease

Thulasi Bikku, Alice Chimhondoro, Preeti Yadav, Vijaya Lakshmi Dwarakacherla et al.
Mathematics Open
Artificial Intelligence in Healthcare
article

Leakage-Aware and Reproducible Comparison of Eight Machine Learning Classifiers for the Diagnostic Classification of Heart Disease

Thulasi Bikku, Alice Chimhondoro, Preeti Yadav, Vijaya Lakshmi Dwarakacherla, B. Sahith, Krishna Babu Kamma
article en

Abstract

Machine-learning studies of heart disease frequently report near-perfect accuracy on the Cleveland benchmark; however, direct comparison of results is hindered by inconsistencies in preprocessing, data partitioning and evaluation metrics across studies. We present a leakage-aware comparison of eight classifiers (logistic regression, naive Bayes, support vector machine, k-nearest neighbours, decision tree, random forest, gradient boosting and a multilayer perceptron) for the diagnostic classification of angiographically defined coronary artery disease. A widely circulated 1,025-record file labelled as Cleveland data contains more records than the combined total of patients in the four UCI heart-disease cohorts (920) and therefore necessarily includes repeated records. All analyses were therefore repeated on the original Cleveland cohort (n = 303, 139 with disease) with ten repetitions of stratified 10-fold cross-validation, and with imputation, scaling, encoding and outlier handling fitted inside each training fold. ROC–AUC was computed from predicted probabilities, and calibration, tuning, feature availability, sex-specific performance and external validation on the Hungarian cohort (n = 294) were examined. Logistic regression had the highest mean AUC (0.911 ± 0.051; accuracy 0.842 ± 0.061), but after Holm correction it could not be separated from random forest, SVM, the multilayer perceptron, knearest neighbours or naive Bayes; only the decision tree and gradient boosting were inferior. Adding 722 repeated records to the unique patients raised random forest accuracy from 0.820 to 0.992, which reproduces the near-perfect accuracy (0.997) obtained with the 1,025-record file. Without ca and thal, the AUC of logistic regression was 0.876, and a resting-variable set gave 0.842. External AUC values ranged from 0.867 to 0.886 for six of the eight classifiers. The results pertain to the diagnostic classification of prevalent disease in referred patients rather than prospective risk prediction, and establish a transparent methodological baseline rather than a model intended for clinical deployment.

Mathematics Open
Artificial Intelligence in Healthcare
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon