Label Provenance and the Temporal Degradation of ICU Prediction Models: A Preregistered Equivalence Study

Background: Intensive care prediction labels come from rules that read different sources: a creatinine assay, a documented decision to culture and treat, a billing code. We asked whether models trained on labels that depend more on documented clinician action age faster. Methods: Holding cohort, features and task fixed, we ordered four labels by action dependence, transferred three model families from 2008–2013 to three later epochs of MIMIC-IV (51,604 stays) and compared each transferred model with a within-epoch ora-cle. The preregistered estimand, the difference between two labels transfer gaps, was judged against a smallest effect size of interest of 0.010 in the area under the receiver oper-ating characteristic curve. Results: With both tests Holm-adjusted, 4 of 12 estimates are practically equivalent, 6 inconclusive and 2 meaningful in the opposite direction. In 2020–2022, the model for post-hour-6 Sepsis-3 onset aged less than the Sequential Organ Failure Assessment model sharing its trunk (difference −0.0208 [−0.0297, −0.0114], full training set; −0.0227, matched). Its sign persisted in all further checks. Across 168 eICU hospitals, the acute kidney injury contrast is practically equivalent under random hospital splits (−0.0014, full training) but ranges from +0.0481 to −0.0207 by held-out region. Conclusions: The primary analysis found no evidence that models trained on more action-dependent labels age faster.

Authors

Institutions

Publication Details

Journal
BioMedInformatics
Published
2026-11-17
DOI
https://doi.org/10.3390/biomedinformatics6060110
Primary Topic
Machine Learning in Healthcare
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Label Provenance and the Temporal Degradation of ICU Prediction Models: A Preregistered Equivalence Study

Anna Pavlovna Bakanova, Arkadiusz Gola, Saule Askerbekovna Belginova, Aigerim Melsatovna Ismukhamedova et al.
1 links
BioMedInformatics
Machine Learning in Healthcare
article

Label Provenance and the Temporal Degradation of ICU Prediction Models: A Preregistered Equivalence Study

Anna Pavlovna Bakanova, Arkadiusz Gola, Saule Askerbekovna Belginova, Aigerim Melsatovna Ismukhamedova, Aisat Khalimov
article en

Abstract

Background: Intensive care prediction labels come from rules that read different sources: a creatinine assay, a documented decision to culture and treat, a billing code. We asked whether models trained on labels that depend more on documented clinician action age faster. Methods: Holding cohort, features and task fixed, we ordered four labels by action dependence, transferred three model families from 2008–2013 to three later epochs of MIMIC-IV (51,604 stays) and compared each transferred model with a within-epoch ora-cle. The preregistered estimand, the difference between two labels transfer gaps, was judged against a smallest effect size of interest of 0.010 in the area under the receiver oper-ating characteristic curve. Results: With both tests Holm-adjusted, 4 of 12 estimates are practically equivalent, 6 inconclusive and 2 meaningful in the opposite direction. In 2020–2022, the model for post-hour-6 Sepsis-3 onset aged less than the Sequential Organ Failure Assessment model sharing its trunk (difference −0.0208 [−0.0297, −0.0114], full training set; −0.0227, matched). Its sign persisted in all further checks. Across 168 eICU hospitals, the acute kidney injury contrast is practically equivalent under random hospital splits (−0.0014, full training) but ranges from +0.0481 to −0.0207 by held-out region. Conclusions: The primary analysis found no evidence that models trained on more action-dependent labels age faster.

BioMedInformaticsVol. 6(6)
Turan University (KZ), D. Serikbayev East Kazakhstan State Technical University (KZ), Kazakh-American Free University (KZ), Lublin University of Technology (PL)
Good health and well-being
Machine Learning in Healthcare
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon