Temporal leakage and evaluation-boundary effects in heart failure mortality prediction: a unified reanalysis of the 299-patient heart failure clinical records dataset
Background The 299-patient Heart Failure Clinical Records dataset is widely used for mortality prediction, yet reported performance varies substantially. Differences in prediction time, follow-up handling, resampling, and data splitting may materially affect apparent model performance. Objective To examine how prediction-time alignment, modelling procedures, and evaluation boundaries influence performance estimates on this shared benchmark, using a targeted methodological audit and harmonized reanalysis of representative modelling configurations. Methods Four representative modelling routes were evaluated using the same dataset. Admission-time classification used 11 baseline variables and stratified 10-fold cross-validation, with applicable resampling and feature-construction procedures restricted to the training data. A parsimonious random forest using ejection fraction and serum creatinine was also assessed. Follow-up duration was excluded from baseline classification and retained for time-to-event analyses. Sensitivity analyses examined follow-up-duration inclusion and changes in preprocessing boundaries. Patient-level bootstrap confidence intervals, repeated-seed analyses, calibration, and exploratory decision-curve analysis were additionally performed. Results Fold-mean AUROCs across the four 11-variable classification configurations ranged from 0.724 to 0.784. The two-feature random forest achieved a pooled out-of-fold AUROC of 0.779 (95% CI, 0.721–0.832), with a paired difference of 0.007 (95% CI, −0.047 to 0.062) from the 11-variable random-forest comparator. Adding follow-up duration increased fold-mean AUROC by 0.121–0.135 within matched predictor sets. The two-feature model retained substantial discrimination but showed limited calibration, with a calibration slope of 0.234 (95% CI, 0.154–0.404). Preprocessing sensitivity analyses also produced substantial variation in apparent performance. Conclusions Prediction-time alignment and evaluation procedures materially influenced performance estimates on this benchmark. Similar discrimination did not imply comparable probability calibration or equivalent patient-level risk estimates. Clinical interpretation therefore requires explicit definition of prediction time, outcome horizon, model implementation, and evaluation procedures, followed by validation in independent contemporary cohorts.
Authors
- 刁一娜 (ORCID: https://orcid.org/0009-0002-8119-9420)
- Wenzeng Wang
- Kuikui Zhang
Institutions
- Heilongjiang University of Chinese Medicine (CN)
- First Affiliated Hospital of Heilongjiang University of Chinese Medicine (CN)
Publication Details
- Journal
- Frontiers in Cardiovascular Medicine
- Published
- 2026-11-16
- DOI
- https://doi.org/10.3389/fcvm.2026.1980349
- Primary Topic
- Artificial Intelligence in Healthcare
- Type
- article
- Field-Weighted Citation Impact
- 0.00