Leakage-Aware Preprocessing and Out-of-Fold Threshold Selection in Benchmark Tabular Classification: A Reproducible Six-Workflow Study on Iris, Customer Churn, and Equity Price Data

Benchmark tabular datasets are routinely used to teach and demonstrate supervised machine learning, yet the workflows built on them are rarely reported in enough detail to be reproduced or audited. Two failure modes recur: information leakage during preprocessing and model selection, and decision thresholds selected on the same data used to report performance. This paper reports a fully reproducible, leakage-audited set of six workflows executed over six supplied data files: Fisher's Iris data (150 records), the BigML/Orange customer-churn benchmark (3,333 records supplied as a fixed 80/20 train-test partition), and a 497,472-row daily equity price table used to exercise time-aware preprocessing, alongside two further files that are audited but not modelled. We document a dataset and data-quality audit, then evaluate k-nearest neighbours, an unrestricted and cost-complexity-pruned decision tree, L2-regularised logistic regression, a tuned random forest, and linear and RBF support vector machines under one protocol: all imputation, encoding, scaling, hyper-parameter search, and threshold selection are fitted inside training folds only, and the held-out partition is scored exactly once per configuration. Headline held-out results are KNN 0.933 accuracy / 0.933 macro-F1 on Iris, pruned tree 0.967 accuracy / 0.967 macro-F1 on Iris, logistic regression ROC-AUC 0.827 with out-of-fold threshold 0.64, random forest ROC-AUC 0.926 with F1 0.836 and Brier score 0.041, and RBF SVM ROC-AUC 0.913 with F1 0.697. We show that collinearity pruning reduced the maximum variance inflation factor in the churn design matrix from 1.04 x 10^7 to 1.00, that impurity-based and permutation importances disagree substantially, that seed variation moves random-forest F1 by about two points while AUC is stable to within 0.002, and that out-of-fold threshold selection trades a large share of recall for precision on an imbalanced task. We release code, frozen datasets, trained pipelines, executed notebooks, and all figures and tables. We also state plainly what this study does not establish: the datasets are public and small, Iris results are near-ceiling and pedagogically useful rather than novel, and none of the reported models should be read as clinically or commercially deployable.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-29
DOI
https://doi.org/10.5281/zenodo.23766239
Primary Topic
Machine Learning and Data Classification
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Leakage-Aware Preprocessing and Out-of-Fold Threshold Selection in Benchmark Tabular Classification: A Reproducible Six-Workflow Study on Iris, Customer Churn, and Equity Price Data

Emmanuel Tolulope Ademola
4 links
Zenodo (CERN European Organization for Nuclear Research)
Machine Learning and Data Classification
preprint

Leakage-Aware Preprocessing and Out-of-Fold Threshold Selection in Benchmark Tabular Classification: A Reproducible Six-Workflow Study on Iris, Customer Churn, and Equity Price Data

Emmanuel Tolulope Ademola
preprint en

Abstract

Benchmark tabular datasets are routinely used to teach and demonstrate supervised machine learning, yet the workflows built on them are rarely reported in enough detail to be reproduced or audited. Two failure modes recur: information leakage during preprocessing and model selection, and decision thresholds selected on the same data used to report performance. This paper reports a fully reproducible, leakage-audited set of six workflows executed over six supplied data files: Fisher's Iris data (150 records), the BigML/Orange customer-churn benchmark (3,333 records supplied as a fixed 80/20 train-test partition), and a 497,472-row daily equity price table used to exercise time-aware preprocessing, alongside two further files that are audited but not modelled. We document a dataset and data-quality audit, then evaluate k-nearest neighbours, an unrestricted and cost-complexity-pruned decision tree, L2-regularised logistic regression, a tuned random forest, and linear and RBF support vector machines under one protocol: all imputation, encoding, scaling, hyper-parameter search, and threshold selection are fitted inside training folds only, and the held-out partition is scored exactly once per configuration. Headline held-out results are KNN 0.933 accuracy / 0.933 macro-F1 on Iris, pruned tree 0.967 accuracy / 0.967 macro-F1 on Iris, logistic regression ROC-AUC 0.827 with out-of-fold threshold 0.64, random forest ROC-AUC 0.926 with F1 0.836 and Brier score 0.041, and RBF SVM ROC-AUC 0.913 with F1 0.697. We show that collinearity pruning reduced the maximum variance inflation factor in the churn design matrix from 1.04 x 10^7 to 1.00, that impurity-based and permutation importances disagree substantially, that seed variation moves random-forest F1 by about two points while AUC is stable to within 0.002, and that out-of-fold threshold selection trades a large share of recall for precision on an imbalanced task. We release code, frozen datasets, trained pipelines, executed notebooks, and all figures and tables. We also state plainly what this study does not establish: the datasets are public and small, Iris results are near-ceiling and pedagogically useful rather than novel, and none of the reported models should be read as clinically or commercially deployable.

Zenodo (CERN European Organization for Nuclear Research)
Adekunle Ajasin University (NG)
Machine Learning and Data Classification
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon