Retrieval Augmented Cross-Domain LifeLong Behavior Modeling for Enhancing Click-through Rate Prediction

Lifelong behavior modeling for single-domain has been widely investigated in industry click-through (CTR) prediction. However, some domains do not always have rich historical behaviors in online platforms, so cross-domain lifelong behavior modeling is overlooked. This paper proposes a novel retrieval augmented lifelong cross-domain net (RAL-CDNet) to address the challenges in cross-domain lifelong behavior modeling. There are three components in RAL-CDNet, i.e., cross-domain retrieval unit, cross-domain alignment unit, and cross-net. As the general search unit in the previous study, a cross-domain retrieval unit features a retrieval augmented paradigm that utilizes a pre-trained language model to learn the intrinsic textual information of user behaviors and generates the sequential behaviors from the source domain based on sequential behaviors in the target domain. The retrieval augmented behaviors can achieve consistency and capture accurate hidden interest for target domain CTR prediction. Furthermore, we propose the cross-domain alignment unit to align the embeddings across domains by adding a semantic-guided contrastive loss and auxiliary task loss in the source domain. This allows the embeddings to be consistent across domains and have enough source information to capture the cross-domain relation. Finally, the cross-net utilizes two-level attention techniques to enhance the final prediction in the target domain. We conduct extensive experiments on both a public dataset and an industrial dataset from the WeChat advertising platform to demonstrate the effectiveness of RAL-CDNet in terms of offline and online metrics.