Machine Learning on Small Experimental Datasets in Manufacturing: A Review of Data Leakage, Grouped Validation, and Uncertainty Quantification

Machine learning is increasingly used to predict machining and process quality from experimental data, yet such data are typically small, generated by designed experiments, and hierarchically organized, with several measurements taken on the same specimen, tool, or batch. Under these conditions, standard evaluation practice can substantially overstate predictive performance. This review synthesizes the methodological literature from statistics, machine learning, and applied domains to provide guidance for small-data machine learning in manufacturing. We present a taxonomy of data-leakage pathways, compare validation strategies (row-wise, grouped, blocked, nested, and bootstrap-based), discuss model families suited to small tabular data, and review uncertainty-quantification methods, including conformal prediction for grouped data and statistical procedures for comparing models. A reproducible simulation of a specimen–section design shows that row-wise cross-validation increasingly underestimates the error on new specimens as within-specimen correlation grows—whereas grouped cross-validation remains close to it—and that non-nested model selection yields optimistic estimates. Additional experiments with constant total variance, a full-factorial design, and six model families show that the bias is driven by within-specimen dependence and is largest for flexible learners. The review concludes with a reporting checklist and open research directions.