The LambdaGap Framework for Precision-Oriented Ranking

LambdaRank has proven effective for optimizing information retrieval metrics such as Normalized Discounted Cumulative Gain (NDCG). However, its application to Precision at document k (P@k) poses significant challenges because of the metric’s unique definition, which heavily restricts the number of effective training document pairs. This limitation diminishes the learning signal for relevant documents beyond the top k, potentially resulting in suboptimal performance. To overcome this, we propose LambdaGap, a ranking algorithm inspired by LambdaRank specifically tailored for optimizing P@k. LambdaGap replaces the pairwise weighting scheme in LambdaRank by one where pairs of documents within k positions in the ranking are masked out. We establish a theoretical link between LambdaGap and P@k by identifying the implicit metric optimized by the model. Furthermore, we introduce a new metric, Average Relevance Position beyond document k, which can be used in conjunction with LambdaRank to indirectly optimize for P@k. Our extensive experiments on publicly available datasets demonstrate the effectiveness of the proposed methods, yielding statistically significant improvements in P@k performance and highlighting their potential for more efficient training.