An Effective Retrieval Method to Improve RAG Performance

Although large language models (LLMs) have demonstrated impressive capabilities in generating coherent and fluent text, they often produce irrelevant content when tasked with specialized domains. To address these challenges, Retrieval-Augmented Generation (RAG) combines retrieval and generation processes to enhance the relevance, accuracy, and diversity of LLM responses. However, naive RAG approaches often struggle with precision and recall during the retrieval phase, leading to the selection of misaligned or irrelevant chunks, and sometimes missing critical information. In this work, we propose a novel approach that integrates both word-level and sentence-level retrieval techniques to optimize the retrieval process. By improving the alignment of retrieved information, our method addresses these precision and recall issues more effectively than traditional retrieval approaches. Extensive experiments on benchmark datasets show that our method is significantly superior to existing retrieval strategies, reducing computational overhead and improving accuracy. Our results not only highlight the effectiveness of advanced retrieval strategies in improving LLM performance but also demonstrate their practical implications for scaling NLP systems in real-world applications.