Temporal leakage and evaluation-boundary effects in heart failure mortality prediction: a unified reanalysis of the 299-patient heart failure clinical records dataset

Background The 299-patient Heart Failure Clinical Records dataset is widely used for mortality prediction, yet reported performance varies substantially. Differences in prediction time, follow-up handling, resampling, and data splitting may materially affect apparent model performance. Objective To examine how prediction-time alignment, modelling procedures, and evaluation boundaries influence performance estimates on this shared benchmark, using a targeted methodological audit and harmonized reanalysis of representative modelling configurations. Methods Four representative modelling routes were evaluated using the same dataset. Admission-time classification used 11 baseline variables and stratified 10-fold cross-validation, with applicable resampling and feature-construction procedures restricted to the training data. A parsimonious random forest using ejection fraction and serum creatinine was also assessed. Follow-up duration was excluded from baseline classification and retained for time-to-event analyses. Sensitivity analyses examined follow-up-duration inclusion and changes in preprocessing boundaries. Patient-level bootstrap confidence intervals, repeated-seed analyses, calibration, and exploratory decision-curve analysis were additionally performed. Results Fold-mean AUROCs across the four 11-variable classification configurations ranged from 0.724 to 0.784. The two-feature random forest achieved a pooled out-of-fold AUROC of 0.779 (95% CI, 0.721–0.832), with a paired difference of 0.007 (95% CI, −0.047 to 0.062) from the 11-variable random-forest comparator. Adding follow-up duration increased fold-mean AUROC by 0.121–0.135 within matched predictor sets. The two-feature model retained substantial discrimination but showed limited calibration, with a calibration slope of 0.234 (95% CI, 0.154–0.404). Preprocessing sensitivity analyses also produced substantial variation in apparent performance. Conclusions Prediction-time alignment and evaluation procedures materially influenced performance estimates on this benchmark. Similar discrimination did not imply comparable probability calibration or equivalent patient-level risk estimates. Clinical interpretation therefore requires explicit definition of prediction time, outcome horizon, model implementation, and evaluation procedures, followed by validation in independent contemporary cohorts.

Authors

Institutions

Publication Details

Journal
Frontiers in Cardiovascular Medicine
Published
2026-11-16
DOI
https://doi.org/10.3389/fcvm.2026.1980349
Primary Topic
Artificial Intelligence in Healthcare
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Temporal leakage and evaluation-boundary effects in heart failure mortality prediction: a unified reanalysis of the 299-patient heart failure clinical records dataset

刁一娜, Wenzeng Wang, Kuikui Zhang
2 links
Frontiers in Cardiovascular Medicine
Artificial Intelligence in Healthcare
article

Temporal leakage and evaluation-boundary effects in heart failure mortality prediction: a unified reanalysis of the 299-patient heart failure clinical records dataset

刁一娜, Wenzeng Wang, Kuikui Zhang
article en

Abstract

Background The 299-patient Heart Failure Clinical Records dataset is widely used for mortality prediction, yet reported performance varies substantially. Differences in prediction time, follow-up handling, resampling, and data splitting may materially affect apparent model performance. Objective To examine how prediction-time alignment, modelling procedures, and evaluation boundaries influence performance estimates on this shared benchmark, using a targeted methodological audit and harmonized reanalysis of representative modelling configurations. Methods Four representative modelling routes were evaluated using the same dataset. Admission-time classification used 11 baseline variables and stratified 10-fold cross-validation, with applicable resampling and feature-construction procedures restricted to the training data. A parsimonious random forest using ejection fraction and serum creatinine was also assessed. Follow-up duration was excluded from baseline classification and retained for time-to-event analyses. Sensitivity analyses examined follow-up-duration inclusion and changes in preprocessing boundaries. Patient-level bootstrap confidence intervals, repeated-seed analyses, calibration, and exploratory decision-curve analysis were additionally performed. Results Fold-mean AUROCs across the four 11-variable classification configurations ranged from 0.724 to 0.784. The two-feature random forest achieved a pooled out-of-fold AUROC of 0.779 (95% CI, 0.721–0.832), with a paired difference of 0.007 (95% CI, −0.047 to 0.062) from the 11-variable random-forest comparator. Adding follow-up duration increased fold-mean AUROC by 0.121–0.135 within matched predictor sets. The two-feature model retained substantial discrimination but showed limited calibration, with a calibration slope of 0.234 (95% CI, 0.154–0.404). Preprocessing sensitivity analyses also produced substantial variation in apparent performance. Conclusions Prediction-time alignment and evaluation procedures materially influenced performance estimates on this benchmark. Similar discrimination did not imply comparable probability calibration or equivalent patient-level risk estimates. Clinical interpretation therefore requires explicit definition of prediction time, outcome horizon, model implementation, and evaluation procedures, followed by validation in independent contemporary cohorts.

Frontiers in Cardiovascular MedicineVol. 13
Heilongjiang University of Chinese Medicine (CN), First Affiliated Hospital of Heilongjiang University of Chinese Medicine (CN)
Good health and well-being
Artificial Intelligence in Healthcare
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon