Overview of leakage scenarios in supervised machine learning

Abstract Machine learning (ML) provides powerful tools for predictive modeling. ML’s popularity stems from the promise of sample-level prediction with applications across a variety of fields from physics and marketing to healthcare. However, if not properly implemented and evaluated, ML pipelines may contain leakage typically resulting in overoptimistic performance estimates and failure to generalize to new data. This can have severe negative financial and societal implications. Our aim is to expand understanding associated with causes leading to leakage when designing, implementing, and evaluating ML pipelines. Illustrated by concrete examples, we provide a comprehensive overview and discussion of various types of leakage that may arise in ML pipelines.

Authors

Institutions

Publication Details

Journal
Journal Of Big Data
Published
2025-05-29
DOI
https://doi.org/10.1186/s40537-025-01193-8
Citations
25
Primary Topic
Anomaly Detection Techniques and Applications
Type
article
Field-Weighted Citation Impact
62.02

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Overview of leakage scenarios in supervised machine learning

Eliana Nicolaisen‐Sobesky, Simon Bodo Eickhoff, Michael Götz, Juha M. Lahnakoski et al.
25 citations
Journal Of Big Data
Anomaly Detection Techniques and Applications
62.02
article

Overview of leakage scenarios in supervised machine learning

Eliana Nicolaisen‐Sobesky, Simon Bodo Eickhoff, Michael Götz, Juha M. Lahnakoski, Sami Hamdan, Leonard Sasse, Federico Raimondo, Juergen Dukart, Bradley C. Love, Kaustubh R. Patil, Vera Komeyer, Abhijit Kulkarni
article en
25 citations

Abstract

Abstract Machine learning (ML) provides powerful tools for predictive modeling. ML’s popularity stems from the promise of sample-level prediction with applications across a variety of fields from physics and marketing to healthcare. However, if not properly implemented and evaluated, ML pipelines may contain leakage typically resulting in overoptimistic performance estimates and failure to generalize to new data. This can have severe negative financial and societal implications. Our aim is to expand understanding associated with causes leading to leakage when designing, implementing, and evaluating ML pipelines. Illustrated by concrete examples, we provide a comprehensive overview and discussion of various types of leakage that may arise in ML pipelines.

Journal Of Big DataVol. 12(1)
Los Alamos National Laboratory (US), Indian Institute of Technology Bombay (IN), Forschungszentrum Jülich (DE), Universität Ulm (DE), Max Planck Institute for Human Cognitive and Brain Sciences (DE), Global Services (Slovakia) (SK), University Hospital Ulm (DE), Heinrich Heine University Düsseldorf (DE)
European Commission, Deutsche Forschungsgemeinschaft, Bundesministerium für Bildung und Forschung, Max-Planck-Gesellschaft
Openalex Percentile: Top 0%
Anomaly Detection Techniques and Applications
62.02
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.