An Explainable and Optimized Ensemble Learning Framework Using XGBoost, CatBoost, and LightGBM for Tabular Predictive Analytics

This study presents an explainable and Bayesian-optimized stacking ensemble learning framework for predictive analytics on complex tabular datasets. The proposed framework integrates XGBoost, CatBoost, and LightGBM as heterogeneous base learners, with hyperparameters optimized using Tree-Structured Parzen Estimator (TPE) Bayesian optimization through Optuna. To reduce validation bias and data leakage, the framework employs nested stratified five-fold cross-validation and constructs an out-of-fold probability matrix for an ElasticNet-regularized logistic meta-learner. The framework is evaluated using the UCI Default of Credit Card Clients dataset containing 30,000 records and 23 input variables. On an independent holdout test set of 6,000 observations, the proposed stacking meta-ensemble achieved an accuracy of 82.41%, F1-score of 0.4924, PR-AUC of 0.5612, and ROC-AUC of 0.7891. Statistical evaluation using DeLong's test, the Wilcoxon signed-rank test, and bootstrap confidence intervals indicates that the ensemble provides a statistically significant improvement over the strongest individual learner, CatBoost. Model transparency is provided through TreeSHAP, enabling global feature-importance analysis and local instance-level explanations. The results demonstrate a practical trade-off between predictive performance, interpretability, and computational efficiency for high-stakes tabular predictive analytics.