Evaluating Cross-Validation Reliability in Kaggle Tabular Binary Classification Competitions Using Bayesian Melding

This preprint investigates the reliability of stratified 5-fold cross-validation in Kaggle tabular binary classification competitions. The study evaluates 3,000 model configurations across three completed Kaggle competitions and applies Bayesian melding to integrate cross-validation, public leaderboard, and private leaderboard scores as noisy observations of latent true model performance.