Label validity before model choice: a leakage-aware evaluation of early-warning prediction in enterprise event logs

To examine which early-warning claims remain defensible when event-log features, duration thresholds, and development labels are restricted to information available at deployment. Three public logs (188,944 cases; 1,850,318 events) were split chronologically by case arrival into training, calibration, validation, and test sets. Kaplan–Meier duration thresholds used historical training events only; development prefixes and label-availability times preceded the first test arrival. Logistic regression and LightGBM were compared with a GRU benchmark at event-count checkpoints. Uncertainty used case, arrival-month, circular moving-block, and short-window weekly bootstrap schemes. Eleven of 32 in-scope configurations were evaluable. Loan delay AUC ranged 0.639–0.705, permit repetition AUC 0.530–0.932, and traffic-fine repetition AUC 0.485–0.723. Trajectories describe selected populations, not learning effects; 157 of 562 common permit cases changed labels. Concept additions produced small AUC changes, and the GRU showed no uniform advantage. Moving-block intervals condition on fixed fitted models and therefore do not include model-selection or fitting uncertainty in this final revision. The study provides an auditable reporting protocol that makes label availability, non-evaluable tasks, checkpoint selection, temporal dependence, and the limits of observational guidance explicit before model comparisons are interpreted.

Authors

Publication Details

Journal
Figshare
Published
2026-11-19
DOI
https://doi.org/10.6084/m9.figshare.35058448
Primary Topic
Business Process Modeling and Analysis
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Label validity before model choice: a leakage-aware evaluation of early-warning prediction in enterprise event logs

Sabina-Cristiana Necula
Figshare
Business Process Modeling and Analysis
article

Label validity before model choice: a leakage-aware evaluation of early-warning prediction in enterprise event logs

Sabina-Cristiana Necula
article en

Abstract

To examine which early-warning claims remain defensible when event-log features, duration thresholds, and development labels are restricted to information available at deployment. Three public logs (188,944 cases; 1,850,318 events) were split chronologically by case arrival into training, calibration, validation, and test sets. Kaplan–Meier duration thresholds used historical training events only; development prefixes and label-availability times preceded the first test arrival. Logistic regression and LightGBM were compared with a GRU benchmark at event-count checkpoints. Uncertainty used case, arrival-month, circular moving-block, and short-window weekly bootstrap schemes. Eleven of 32 in-scope configurations were evaluable. Loan delay AUC ranged 0.639–0.705, permit repetition AUC 0.530–0.932, and traffic-fine repetition AUC 0.485–0.723. Trajectories describe selected populations, not learning effects; 157 of 562 common permit cases changed labels. Concept additions produced small AUC changes, and the GRU showed no uniform advantage. Moving-block intervals condition on fixed fitted models and therefore do not include model-selection or fitting uncertainty in this final revision. The study provides an auditable reporting protocol that makes label availability, non-evaluable tasks, checkpoint selection, temporal dependence, and the limits of observational guidance explicit before model comparisons are interpreted.

Figshare
Business Process Modeling and Analysis
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon