A Curated Keyword Store as a Complement to RAG for Closed-Domain Question Answering

Topical noise is added by the dense-vector retrieval of RAG systems. That noise can be avoided, when the knowledge space is small, and maintained actively! The Curated Knowledge Store (CKS): retrieval layer designed out of Markdown file with explicit routing to keywords, human-verified flags and cases of staleness date. The retrieval is deterministic, that is, it will either conform to the vocabulary that the query declares or it won't. This clears out partial-match noise, but it means that no words are recalled on a vocabulary gap. With a CKS-first / RAG-fallback hybrid, that accuracy is leveraged when that's available, and is rerouted to RAG in the event that it isn't. We outline design, analyse the PT curve of our design compared to BM25, dense retrieval, and present a small feasibility study using ten factoid questions from one newspaper edition. The concatenation hybrid achieved 8.7/10, whereas RAG alone achieved 8.0/10, and CKS alone achieved 4.5/10. These figures do not represent actual, but indicative, figures. The principle behind the architecture is the ability to show what is and isn't happening, but not necessarily the accuracy of what is happening.