Accelerated retrieval-augmented generation method based on category matching and partitioned retrieval (CARAG)

Retrieval-augmented large language models (LLMs) have become an effective method for improving the quality of language model responses by integrating information retrieval with language model generation. This approach incorporates non-parametric knowledge from external knowledge bases into LLMs. Many enterprises leverage retrieval-augmented generation techniques to build private knowledge bases using internal documents and resources as external knowledge sources. However, as the volume of data in external knowledge bases increases, the scale of the index also expands, leading to a decrease in retrieval speed. This study proposes a framework based on category matching and partitioned retrieval. By partitioning the database and retrieving only the relevant partitions based on the category of user queries, this method significantly enhances retrieval speed and accuracy while effectively reducing interference from irrelevant content.