purgedcv: Label-Aware Cross-Validation for Overlapping-Horizon Prediction in Python

Cross-validation is routinely used to estimate out-of-sample performance in statistical learning, but standard shuffled or blocked folds can be invalid when responses are measured over future intervals. A label such as the mean demand over the next twelve half-hours, the next-day rainfall amount, or the return over the next twenty bars overlaps the labels of nearby rows. If overlapping label intervals are split between training and test sets, the validation score partly measures information reuse rather than generalization. This article formalizes split-level conditions for leakage-aware validation in overlapping-label time-series and panel data, and presents purgedcv, a Python implementation that exposes purging, embargoing, walk-forward validation, group-purged folds, and combinatorial purged cross-validation through the scikit-learn splitter protocol, with diagnostic assertions for auditing train/test splits. A controlled experiment with an unpredictable target shows that shuffled k-fold can report a mean out-of-sample R2 of 0.918 while admitting complete train/test label overlap. A full-population benchmark on Low Carbon London smart-meter data shows a more nuanced case: the temporal leakage gap is small but measurable, whereas the larger issue is household-level generalization. The software, notebooks, tests, and benchmark scripts are open source and make the validation choice auditable rather than implicit.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-05-21
DOI
https://doi.org/10.5281/zenodo.20323362
Primary Topic
Machine Learning and Data Classification
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

purgedcv: Label-Aware Cross-Validation for Overlapping-Horizon Prediction in Python

Евгений Лазарев
Zenodo (CERN European Organization for Nuclear Research)
Machine Learning and Data Classification
preprint

purgedcv: Label-Aware Cross-Validation for Overlapping-Horizon Prediction in Python

Евгений Лазарев
preprint en

Abstract

Cross-validation is routinely used to estimate out-of-sample performance in statistical learning, but standard shuffled or blocked folds can be invalid when responses are measured over future intervals. A label such as the mean demand over the next twelve half-hours, the next-day rainfall amount, or the return over the next twenty bars overlaps the labels of nearby rows. If overlapping label intervals are split between training and test sets, the validation score partly measures information reuse rather than generalization. This article formalizes split-level conditions for leakage-aware validation in overlapping-label time-series and panel data, and presents purgedcv, a Python implementation that exposes purging, embargoing, walk-forward validation, group-purged folds, and combinatorial purged cross-validation through the scikit-learn splitter protocol, with diagnostic assertions for auditing train/test splits. A controlled experiment with an unpredictable target shows that shuffled k-fold can report a mean out-of-sample R2 of 0.918 while admitting complete train/test label overlap. A full-population benchmark on Low Carbon London smart-meter data shows a more nuanced case: the temporal leakage gap is small but measurable, whereas the larger issue is household-level generalization. The software, notebooks, tests, and benchmark scripts are open source and make the validation choice auditable rather than implicit.

Zenodo (CERN European Organization for Nuclear Research)
Machine Learning and Data Classification
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.