Effect of Data Preprocessing Decisions on Sepsis-3 Labels in MIMIC-III and eICU-CRD: A Sensitivity Analysis
Objective: Published Sepsis-3 detection rates range from 3.4 % to 65.2 % in MIMIC-III and from 9.8 % to 47.9 % in eICU-CRD. We tested whether data preprocessing decisions alone produce differences of this magnitude when data, cohort and sepsis definition are held fixed. Materials and Methods: Using the MIMIC-III and eICU-CRD databases we developed and applied a configurable pipeline (medalabel) that stepwise maps the databases into the OMOP Common Data Model, standardizes and cleans the data, and preprocesses them to compute Sepsis-3 labels. Each setting of the preprocessing represents one decision of published Sepsis-3 implementations, and settings can be freely combined. On fixed cohorts of first ICU stays (13,388 in MIMIC-III, 94,782 in eICU-CRD), we changed one setting of a reference configuration at a time (51 variants in five preprocessing domains) and compared detection rates and septic stays (Jaccard index) with the reference. We approximately reconstructed six published pipelines and combined the settings with the largest effects in pairs to test whether their effects add up. The time a value is measured can differ from the time it is documented and thus available for automated analysis. Beyond the usual retrospective view based on measurement time, the pipeline optionally combines both time points to simulate a real-time system. Results: The reference configuration labelled 31.5 % of ICU stays in MIMIC-III and 8.2 % in eICU-CRD as septic. Changing a single setting resulted in sepsis detection rates of 15.3-58.6 % in MIMIC-III and 4.6-17.2 % in eICU-CRD. The largest shifts from the reference, all in the same direction in both databases, came from the SOFA baseline, up to +27.1 percentage points (pp) in MIMIC-III and +9.0 pp in eICU-CRD, from disabling imputation (-16.3 and -2.5 pp), the temporal window (+12.6 and +5.3 pp) and aggregation (-10.1 and -0.8 pp); clinician-validated parameter sets had little effect (-4.8 and -1.1 pp). Even variants that changed the rate by less than 2 pp labelled partly different patients, with Jaccard indices as low as 0.67 in MIMIC-III and 0.56 in eICU-CRD (1 = identical septic stays). Reconstructions deviated from the published rates by 2-10 pp. Two settings applied together deviated from the sum of their single effects by 0.9-11.8 pp in MIMIC-III and by up to 4.9 pp in eICU-CRD. Simulating a real-time system lowered the MIMIC-III rate by 5.0 pp and delayed the label in 45 % of the stays that remained septic. Discussion: With the infection criterion held fixed, preprocessing decisions alone can produce a considerable part of the variation between published sepsis detection rates. Similar rates can stem from different patients, which makes studies and prediction models difficult to compare, and retrospective labels rely on values that a real-time system would not yet have. Conclusion: Sepsis-3 labels from health records are only comparable if their data preprocessing is known. Configurable pipelines allow quantifying the effect of these decisions, computing alternative labels on the same data and, in future, validating them against clinician-adjudicated cases.
Authors
- Anatol‐Fiete Näher (ORCID: https://orcid.org/0000-0002-7908-0175)
- Andreas Edel (ORCID: https://orcid.org/0000-0002-9951-7223)
- Severin Kohler (ORCID: https://orcid.org/0000-0002-7718-6187)
- Jon Barrenetxea (ORCID: https://orcid.org/0000-0002-5466-553X)
- Oliver Kumpf (ORCID: https://orcid.org/0000-0001-7891-8872)
- Felix Balzer (ORCID: https://orcid.org/0000-0003-1575-2056)
- Elias Grünewald (ORCID: https://orcid.org/0000-0001-9076-9240)
- Falk Meyer-Eschenbach (ORCID: https://orcid.org/0009-0000-9813-0249)
- Nicolas Frey (ORCID: https://orcid.org/0009-0000-0195-6096)
- Johannes v. Stoephasius (ORCID: https://orcid.org/0009-0006-8615-4634)
- Fabian Kohlbacher
Institutions
- Hasso Plattner Institute (DE)
- Cal Poly Humboldt (US)
- University Hospital Heidelberg (DE)
- Humboldt-Universität zu Berlin (DE)
- Berlin Institute of Health at Charité - Universitätsmedizin Berlin (DE)
- Freie Universität Berlin (DE)
- Charité - Universitätsmedizin Berlin (DE)
Publication Details
- Journal
- medRxiv
- Published
- 2026-11-15
- DOI
- https://doi.org/10.64898/2026.11.12.26367959
- Primary Topic
- Sepsis Diagnosis and Treatment
- Type
- preprint