An Experimental Analysis of Data Leakage in Machine Learning: Impact on Model Performance and Prevention Techniques

Data leakage is a methodological error in machine learning in which information unavailable at the intended prediction time, or information from the evaluation data, influences model development. It can produce misleading estimates of generalization performance and weaken the reproducibility of research. This paper studies data leakage, its common forms, its impact on model evaluation, and practical prevention methods. A controlled educational experiment is implemented using the Wisconsin Diagnostic Breast Cancer dataset available through scikit-learn. Logistic Regression and Random Forest are compared under a correct workflow, in which scaling and feature selection are fitted only on training data, and a deliberately leaky workflow, in which preprocessing and supervised feature selection are fitted before the train-test split. Across five stratified 75:25 splits, mean accuracy for Logistic Regression was 94.41% with the correct workflow and 93.99% with the leaky workflow; for Random Forest, mean accuracy was 93.71% and 93.43%, respectively. These results illustrate that leakage is a methodological flaw, but it does not guarantee that every metric will increase in every experiment: its effect depends on the dataset, leakage mechanism, split, and model. The paper concludes with a practical checklist for leakage prevention and reproducible reporting.