Predictability Is Not Profitability: An Explainable, Cost-Sensitive Evaluation of Customer Churn Across Three Sectors

Churn prediction research is largely accuracy-oriented, and the link between prediction and financial decision-making remains underdeveloped. This study builds and tests the chain from prediction to profit as follows: calibrated prediction, SHAP-based explanation, a profit-maximizing threshold, and EMP-based evaluation, across five datasets in three sectors. Resampling and class weighting do not improve ranking quality (PR-AUC) and degrade calibration up to 19-fold (ECE); shifting the threshold recovers the same recall without retraining. SHAP-based analysis shows no driver is consistently strong across sectors: usage volume has the broadest reach, but the strongest drivers are dataset-specific, and models do not transfer after semantic alignment. The profit-maximizing threshold matches or beats the fixed 0.5 threshold in all 100 cost-success scenarios examined, and a paired test across datasets and seeds confirms the difference (Wilcoxon p < 0.001); a threshold that looks reasonable by accuracy can still cause a loss. Predictability and profitability rank inversely across datasets (Spearman ρ = −0.90): the most predictable dataset yields the lowest EMP, driven by churners’ value distribution. Compared with ProfLogit, which embeds the profit objective in training, a threshold on a well-calibrated model proves sufficient. Value comes from turning calibrated probabilities into decisions with a financial criterion, not from balancing data.