Label Provenance and the Temporal Degradation of ICU Prediction Models: A Preregistered Equivalence Study
Background: Intensive care prediction labels come from rules that read different sources: a creatinine assay, a documented decision to culture and treat, a billing code. We asked whether models trained on labels that depend more on documented clinician action age faster. Methods: Holding cohort, features and task fixed, we ordered four labels by action dependence, transferred three model families from 2008–2013 to three later epochs of MIMIC-IV (51,604 stays) and compared each transferred model with a within-epoch ora-cle. The preregistered estimand, the difference between two labels transfer gaps, was judged against a smallest effect size of interest of 0.010 in the area under the receiver oper-ating characteristic curve. Results: With both tests Holm-adjusted, 4 of 12 estimates are practically equivalent, 6 inconclusive and 2 meaningful in the opposite direction. In 2020–2022, the model for post-hour-6 Sepsis-3 onset aged less than the Sequential Organ Failure Assessment model sharing its trunk (difference −0.0208 [−0.0297, −0.0114], full training set; −0.0227, matched). Its sign persisted in all further checks. Across 168 eICU hospitals, the acute kidney injury contrast is practically equivalent under random hospital splits (−0.0014, full training) but ranges from +0.0481 to −0.0207 by held-out region. Conclusions: The primary analysis found no evidence that models trained on more action-dependent labels age faster.
Authors
- Anna Pavlovna Bakanova (ORCID: https://orcid.org/0000-0002-8674-3686)
- Arkadiusz Gola (ORCID: https://orcid.org/0000-0002-2935-5003)
- Saule Askerbekovna Belginova (ORCID: https://orcid.org/0000-0002-7238-6016)
- Aigerim Melsatovna Ismukhamedova (ORCID: https://orcid.org/0000-0002-4931-337X)
- Aisat Khalimov (ORCID: https://orcid.org/0009-0001-3019-020X)
Institutions
- Turan University (KZ)
- D. Serikbayev East Kazakhstan State Technical University (KZ)
- Kazakh-American Free University (KZ)
- Lublin University of Technology (PL)
Publication Details
- Journal
- BioMedInformatics
- Published
- 2026-11-17
- DOI
- https://doi.org/10.3390/biomedinformatics6060110
- Primary Topic
- Machine Learning in Healthcare
- Type
- article
- Field-Weighted Citation Impact
- 0.00