An Unbiased Entire-Space Causal Framework for Click-Through Rate Estimation in Pre-Ranking

Online advertising systems increasingly depend on advanced recommendation algorithms to optimize click-through rates (CTR) and maximize revenue. However, these systems frequently grapple with sample selection bias (SSB) at the pre-ranking stage, where CTR estimation models are trained on feedback from the ranking stage but applied to datasets with contrasting distributions. This disparity in data distributions not only leads to inaccurate estimations but also significantly undermines the effectiveness of advertising strategies. Most existing approaches, which rely on empirical label generation and random sampling of unexposed samples, introduce additional distortions in CTR estimation without adequately addressing sample biases. To tackle the SSB issue in the pre-ranking stage, we propose a comprehensive unbiased counterfactual causal framework for CTR estimation. Specifically, to fully leverage the entire- space samples, we introduce a feature-level unbiased label generation module for unexposed samples. Additionally, while modeling the propensity of unexposed samples, we present an unbiased automatic sample selection module that strikes a balance between model accuracy and computational efficiency over large-scale datasets. Furthermore, we formally demonstrate that the proposed framework maintains theoretical unbiasedness across the entire-space. Extensive experimental results validate the superior performance and effectiveness of the proposed framework. Compared to current state-of-the-art (SOTA) empirical and causal methods, our framework achieves a maximum AUC improvement of 4.47% on offline datasets and a 1.11% increase in CTR during online A/B testing. This framework has been successfully deployed in large-scale industrial advertising systems, yielding reliable enhancements in prediction accuracy and revenue outcomes.