DualReward-QPE: Dual Reward-Based Query Preference Enhancement

Information retrieval (IR) systems often suffer from vocabulary mismatches between user queries and relevant documents. Large Language Models (LLMs) provide new opportunities for query expansion (QE), but face challenges such as hallucination and misaligned user intent. This paper introduces DualReward-QPE, a framework that fine-tunes LLMs with Direct Preference Optimization (DPO) while integrating a dual-reward strategy to reduce hallucination and enhance semantic alignment. To construct the preference dataset, the approach combines the large-scale E5 dataset with GPT-3.5 Turbo-generated queries based on the non-factoid question answering dataset ANTIQUE. We jointly employ a rank-based reward and an answer-based reward to guide query expansion generation, resulting in document-guided expansions that can improve dense retrieval performance on MS MARCO and BEIR benchmarks. Experimental results suggest that DualReward-QPE effectively addresses the vocabulary gap and consistently outperforms existing QE methods in both unsupervised and supervised retrieval scenarios.