GaQR: An Efficient Generation-augmented Question Rewriter

Query understanding is an essential part in search systems to improve the recall. Unlike prior works focusing on word expansions, in this paper, we leverage the comprehension ability of LLM to generate detailed queries from a global semantic perspective. To this end, we introduce an efficient GaQR to reformulate a question into several queries using Chain of Thought (CoT) and make it more efficient through knowledge distillation. Specifically, we first prompt a teacher model to generate indicative queries by considering answer generation one step ahead. Then, we filter out low-quality queries by validating the effectiveness of all generated queries in retrieving useful passages. Finally, we distill a student rewriter based on the verified results to improve efficiency. Our experimental results demonstrate that the rewriter improves the retrieval performance by 3% to 15% on the Miracl and NFCorpus datasets and shows good generalisation ability across different retrieval methods. Moreover, the efficiency of the rewriter after knowledge distillation is improved by as much as 5 times. Code is available at https://github.com/youngbeauty250/GaQR.