Hierarchical User Long-term Behavior Modeling for Click-Through Rate Prediction

State-of-the-art approaches for click-through rate (CTR) prediction in industry predominantly rely on transformer-based networks or their variants. However, as user behavior sequences become longer, employing self-attention networks for CTR prediction within a constrained inference time presents a significant challenge. To address this, mainstream methods adopt a classical two-stage paradigm: a General Search Unit (GSU) for quickly retrieving relevant items from long-term behaviors, and an Exact Search Unit (ESU) for applying effective Multi-Head Target Attention (MHTA) over the items selected by the GSU. These two-stage algorithms have certain limitations. Firstly, the GSU needs to retrieve different target subsequences for different target items, restricting the ESU to a suboptimal MHTA network rather than a more effective transformer-based network. Secondly, the GSU retrieves only a subset of items from the user's behavior sequence, ignoring the evolution of user interests and the interrelationships between different points of interest. To overcome these challenges, we propose a novel end-to-end hierarchical user long-term behavior modeling network for CTR prediction (HBM). Specifically, we employ the multi-interest routing layer to channel the user's long-term behavior to several aggregated interest clusters. Furthermore, we introduce a fine interest learning network that selects the top-k interests from the initial aggregated representations. Subsequently, we employ a transformer network to model the user's behavior sequence associated with these top-k interests in a detailed manner, while also capturing the inherent correlations between different user interests at a coarse level. Finally, we integrate the coarse and fine interests. Extensive experiments on two real-world datasets demonstrate the effectiveness of our proposed methods. In addition, an online A/B test on the JD recommendation platform shows promising improvements, with a 2.15% increase in CTR and a 0.98% increase in CVR, accompanied by lower online inference latency.