An Empirical Audit of Performance, Fairness, and Compute Cost in Tabular Hyperparameter Optimization

This study presents a controlled, empirical audit comparing Random Forest and XGBoost optimized via Random Search and Bayesian Optimization across three tabular classification datasets (Adult, Bank Marketing, and Breast Cancer Wisconsin Diagnostic). The audit evaluates predictive performance, group fairness (Demographic Parity Difference, Equal Opportunity Difference), and computational cost under a strict 5-fold nested cross-validation framework with paired statistical testing (Wilcoxon signed-rank, Cohen's d_z). Results demonstrate that XGBoost significantly outperforms Random Forest (balanced accuracy 0.8186 vs. 0.7973, p < 0.001, large effect), while Bayesian HPO yields only marginal gains over Random Search. Fairness disparities on the Adult dataset persist regardless of optimizer choice. A severe runtime anomaly (140× group median) is identified and classified as a system-level execution artifact. Full source code and results are available at: https://github.com/Moazzam9/fair-hpo-tabular-benchmark