Effect of Data Preprocessing Decisions on Sepsis-3 Labels in MIMIC-III and eICU-CRD: A Sensitivity Analysis

Objective: Published Sepsis-3 detection rates range from 3.4 % to 65.2 % in MIMIC-III and from 9.8 % to 47.9 % in eICU-CRD. We tested whether data preprocessing decisions alone produce differences of this magnitude when data, cohort and sepsis definition are held fixed. Materials and Methods: Using the MIMIC-III and eICU-CRD databases we developed and applied a configurable pipeline (medalabel) that stepwise maps the databases into the OMOP Common Data Model, standardizes and cleans the data, and preprocesses them to compute Sepsis-3 labels. Each setting of the preprocessing represents one decision of published Sepsis-3 implementations, and settings can be freely combined. On fixed cohorts of first ICU stays (13,388 in MIMIC-III, 94,782 in eICU-CRD), we changed one setting of a reference configuration at a time (51 variants in five preprocessing domains) and compared detection rates and septic stays (Jaccard index) with the reference. We approximately reconstructed six published pipelines and combined the settings with the largest effects in pairs to test whether their effects add up. The time a value is measured can differ from the time it is documented and thus available for automated analysis. Beyond the usual retrospective view based on measurement time, the pipeline optionally combines both time points to simulate a real-time system. Results: The reference configuration labelled 31.5 % of ICU stays in MIMIC-III and 8.2 % in eICU-CRD as septic. Changing a single setting resulted in sepsis detection rates of 15.3-58.6 % in MIMIC-III and 4.6-17.2 % in eICU-CRD. The largest shifts from the reference, all in the same direction in both databases, came from the SOFA baseline, up to +27.1 percentage points (pp) in MIMIC-III and +9.0 pp in eICU-CRD, from disabling imputation (-16.3 and -2.5 pp), the temporal window (+12.6 and +5.3 pp) and aggregation (-10.1 and -0.8 pp); clinician-validated parameter sets had little effect (-4.8 and -1.1 pp). Even variants that changed the rate by less than 2 pp labelled partly different patients, with Jaccard indices as low as 0.67 in MIMIC-III and 0.56 in eICU-CRD (1 = identical septic stays). Reconstructions deviated from the published rates by 2-10 pp. Two settings applied together deviated from the sum of their single effects by 0.9-11.8 pp in MIMIC-III and by up to 4.9 pp in eICU-CRD. Simulating a real-time system lowered the MIMIC-III rate by 5.0 pp and delayed the label in 45 % of the stays that remained septic. Discussion: With the infection criterion held fixed, preprocessing decisions alone can produce a considerable part of the variation between published sepsis detection rates. Similar rates can stem from different patients, which makes studies and prediction models difficult to compare, and retrospective labels rely on values that a real-time system would not yet have. Conclusion: Sepsis-3 labels from health records are only comparable if their data preprocessing is known. Configurable pipelines allow quantifying the effect of these decisions, computing alternative labels on the same data and, in future, validating them against clinician-adjudicated cases.

Authors

Institutions

Publication Details

Journal
medRxiv
Published
2026-11-15
DOI
https://doi.org/10.64898/2026.11.12.26367959
Primary Topic
Sepsis Diagnosis and Treatment
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Effect of Data Preprocessing Decisions on Sepsis-3 Labels in MIMIC-III and eICU-CRD: A Sensitivity Analysis

Anatol‐Fiete Näher, Andreas Edel, Severin Kohler, Jon Barrenetxea et al.
medRxiv
Sepsis Diagnosis and Treatment
preprint

Effect of Data Preprocessing Decisions on Sepsis-3 Labels in MIMIC-III and eICU-CRD: A Sensitivity Analysis

Anatol‐Fiete Näher, Andreas Edel, Severin Kohler, Jon Barrenetxea, Oliver Kumpf, Felix Balzer, Elias Grünewald, Falk Meyer-Eschenbach, Nicolas Frey, Johannes v. Stoephasius, Fabian Kohlbacher
preprint en

Abstract

Objective: Published Sepsis-3 detection rates range from 3.4 % to 65.2 % in MIMIC-III and from 9.8 % to 47.9 % in eICU-CRD. We tested whether data preprocessing decisions alone produce differences of this magnitude when data, cohort and sepsis definition are held fixed. Materials and Methods: Using the MIMIC-III and eICU-CRD databases we developed and applied a configurable pipeline (medalabel) that stepwise maps the databases into the OMOP Common Data Model, standardizes and cleans the data, and preprocesses them to compute Sepsis-3 labels. Each setting of the preprocessing represents one decision of published Sepsis-3 implementations, and settings can be freely combined. On fixed cohorts of first ICU stays (13,388 in MIMIC-III, 94,782 in eICU-CRD), we changed one setting of a reference configuration at a time (51 variants in five preprocessing domains) and compared detection rates and septic stays (Jaccard index) with the reference. We approximately reconstructed six published pipelines and combined the settings with the largest effects in pairs to test whether their effects add up. The time a value is measured can differ from the time it is documented and thus available for automated analysis. Beyond the usual retrospective view based on measurement time, the pipeline optionally combines both time points to simulate a real-time system. Results: The reference configuration labelled 31.5 % of ICU stays in MIMIC-III and 8.2 % in eICU-CRD as septic. Changing a single setting resulted in sepsis detection rates of 15.3-58.6 % in MIMIC-III and 4.6-17.2 % in eICU-CRD. The largest shifts from the reference, all in the same direction in both databases, came from the SOFA baseline, up to +27.1 percentage points (pp) in MIMIC-III and +9.0 pp in eICU-CRD, from disabling imputation (-16.3 and -2.5 pp), the temporal window (+12.6 and +5.3 pp) and aggregation (-10.1 and -0.8 pp); clinician-validated parameter sets had little effect (-4.8 and -1.1 pp). Even variants that changed the rate by less than 2 pp labelled partly different patients, with Jaccard indices as low as 0.67 in MIMIC-III and 0.56 in eICU-CRD (1 = identical septic stays). Reconstructions deviated from the published rates by 2-10 pp. Two settings applied together deviated from the sum of their single effects by 0.9-11.8 pp in MIMIC-III and by up to 4.9 pp in eICU-CRD. Simulating a real-time system lowered the MIMIC-III rate by 5.0 pp and delayed the label in 45 % of the stays that remained septic. Discussion: With the infection criterion held fixed, preprocessing decisions alone can produce a considerable part of the variation between published sepsis detection rates. Similar rates can stem from different patients, which makes studies and prediction models difficult to compare, and retrospective labels rely on values that a real-time system would not yet have. Conclusion: Sepsis-3 labels from health records are only comparable if their data preprocessing is known. Configurable pipelines allow quantifying the effect of these decisions, computing alternative labels on the same data and, in future, validating them against clinician-adjudicated cases.

medRxiv
Hasso Plattner Institute (DE), Cal Poly Humboldt (US), University Hospital Heidelberg (DE), Humboldt-Universität zu Berlin (DE), Berlin Institute of Health at Charité - Universitätsmedizin Berlin (DE), Freie Universität Berlin (DE), Charité - Universitätsmedizin Berlin (DE)
Sepsis Diagnosis and Treatment
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon