Research on Retrieval-augmented Generation Algorithms for Large Language Models

Large language model in the field of text generation shows excellent language skills, combined with the retrieval of generated algorithm is enhanced, the language model is more accurate and timely in a professional field. Therefore, retrieval enhanced generation (Retrieval Enhanced Generation, RAG) has become an important framework for building large language models, but the processing capacity of simple RAG is limited and highly dependent on the quality of retrieval. Therefore, this paper proposes an optimized RAG, adding additional processing steps in the whole process to improve the relevance and overall quality of optimized replies. Our validation shows that the improved RAG has a significant increase in its ability to perform on large models, and by comparing it with fine-tuning techniques, verifies that the combination of RAG and fine-tuning is better than what can be achieved with simple RAG or fine-tuning.