A Model-agnostic Pre-training Framework for Search Result Diversification

Search result diversification focuses on providing relevant and diverse documents covering different users’ intents. Intuitively, training an effective and stable search result diversification model needs a large amount of training data. Unfortunately, annotating such data that encompass real users’ search intents is expensive and time-consuming, and most existing models are trained with limited training data, which might lead to unsatisfactory ranking results. Given that Wikipedia contains massive amounts of rigorous editorial and well-structured data, in this article, we propose a pre-training framework leveraging the large-scale Wikipedia data to build weak-supervised signals. Specifically, we introduce four strategies to extract paired supervised signals reflecting the subtopic coverage information from Wikipedia. We also propose a subtopic-disentangled negative sampling strategy to sample hard negative samples and enhance the model’s ability to identify subtle subtopic differences. Four auxiliary tasks are devised to pre-train the Transformer model, which is further adopted as the representation generation model in the downstream diversified ranking. Experimental results demonstrate that our pre-trained model can significantly improve the performance of several existing models, which confirms the effectiveness and scalability of pre-training for search result diversification.