Human-in-the-loop knowledge base upkeep for retrieval augmented generation applications

Retrieval augmented generation (RAG) has been proposed as a way of grounding the large language models (LLMs) with factual and updated sources of information. However, the quality of the chunks in the knowledge base is important. In this work, we propose a human-in-the-loop knowledge base visualization and upkeep system that aims to maintain the quality of the knowledge base by preassigning tags to chunks based on their structural forms, rather than semantics. This allows a human reviewer to quickly identify pieces of chunks that could be considered noise, and easily remove unwanted chunks that will naturally come, due to the process of document chunking in RAG.