An Interpretable Hybrid of Boosted Trees and Regularised Logistic Regression for Predicting Customer Churn in Online Retail

Churn in online retail is gradual and non-contractual, and the behavioural records used to predict it tend to combine strong monotone effects with a few saturating ones. Linear models capture the first kind of effect and tree ensembles capture the second, yet churn studies usually choose between them. This study proposes a hybrid that blends, on the log-odds scale, a seed-bagged Categorical Boosting (CatBoost) model with an L1- or L2-regularised logistic regression. Both components are tuned by a Tree-structured Parzen Estimator (TPE) on shared inner folds, and the blend weight and decision threshold are selected from inner out-of-fold predictions. Because the blend is linear in log-odds, the Shapley Additive exPlanations (SHAP) of the two components combine in closed form into a single additive decomposition of the hybrid score, so each explanation can be split into a logistic and a boosted contribution, each the product of the selected weight and the corresponding component attribution. The hybrid was compared with fifteen models, including a tuned logistic regression, a tuned Light Gradient Boosting Machine (LightGBM) and an Explainable Boosting Machine (EBM) given the same search budget, under nested cross-validation with 25 outer test folds. It obtained a mean area under the precision–recall curve (PR-AUC) of 0.4658 and a mean area under the receiver operating characteristic curve (ROC-AUC) of 0.8710, the highest of all models on both measures. With the corrected resampled t-test and Holm adjustment for fifteen comparisons, its PR-AUC advantage was significant against 11 rivals, including the tuned LightGBM (+0.0404) and its own seed-bagged CatBoost component (+0.0197); the advantages over the tuned EBM (+0.0143) and the tuned logistic regression (+0.0011, p = 0.6697) were not significant. On the primary endpoint, the hybrid is therefore level with a well-specified logistic regression rather than ahead of it, and what it adds is the separation of each prediction into its logistic and its boosted part. The mean selected weight of the CatBoost component was 0.2240. The attributions assigned a customer attribute without behavioural meaning, used as a negative control, a mean absolute SHAP value of 0.0092, and attribution rankings across outer folds had a mean Spearman correlation of 0.9461 with the pooled ranking. In this dataset, most of the predictive signal is carried by a linear function on the log-odds scale, the boosted component enters the blend with the smaller coefficient in every fold and its increment over the logistic component is not separable from zero in the outer folds, and the hybrid makes that division explicit for every prediction; whether the same balance holds for other retailers, labelling rules and periods is not established by a single dataset and remains open.