Boosting Retrieval-Augmented Generation with Generation-Augmented Retrieval: A Co-Training Approach

Large language models (LLMs) have shown success in knowledge-intensive tasks, including closed-book question answering and entity linking. However, their susceptibility to hallucination undermines their reliability. Retrieval-augmented generation (RAG) partially addresses this issue by combining a retriever to locate relevant documents and a generator to produce responses grounded in the retrieved evidence. Despite its advantages, RAG faces challenges: (i) the structural gap between traditional dense retrievers and autoregressive generators, and (ii) limited generation performance due to insufficient contextual guidance returned by the retriever. To tackle these limitations, we propose MINT, a novel framework that enhances RAG by co-training Retrieval-augMented generatIon and geNeration-augmented reTrieval (GAR). MINT (i) bridges the gap between the retriever and generator using a unified encoder-decoder structure (ii) incorporates an iterative co-training strategy between RAG and GAR, enabling mutual enhancement through pseudo-samples generation, and (iii) introduces three heuristic inference strategies to generate relevant document identifiers and answers. We conduct an empirical study on the KILT benchmark, and MINT is found to yield significant improvements in both retrieval and generation tasks compared with prevailing baselines.