Improving Faithfulness and Mitigating Hallucinations in RAG Systems Through Hierarchical Metadata Retrieval and Prompt Constraint Mechanisms

This paper presents a domain-specific Retrieval-Augmented Generation (RAG) framework that mitigates hallucinations and is designed to strengthen factual consistency in large language model (LLM)-based question-answering systems.Here, hallucination is operationally defined in this study as the generation of an unsupported substantive response to an unanswerable question.Existing RAG frameworks may produce less reliable responses due to retrieval failures or noisy retrieved content, potentially leading to inaccurate or unsupported answers.These limitations are especially problematic in domain-specific applications where factual accuracy is essential, and even minor errors can undermine user trust.To address these issues, this study proposes an enhanced RAG architecture that combines hierarchical metadata retrieval with fact-grounded prompt constraints.Hierarchical metadata retrieval narrows the search space in a structured manner, thereby reducing the amount of irrelevant or noisy information passed to the generation module.Fact-grounded prompt constraints further guide the LLM to generate responses grounded in the retrieved evidence.This study evaluated a Pure LLM, a Basic RAG model, and the proposed model using an internally constructed evaluation set of 210 questions, comprising 105 answerable questions and 105 synthetically generated unanswerable questions.Qwen2.5-14B was used for response generation, and the Ragas framework was used for performance evaluation.Compared with the Basic RAG baseline, the proposed model achieved a higher Faithfulness score (0.8926 vs. 0.7366) among the substantive responses generated by each model.These findings suggest that coordinated control of the retrieval and generation stages may improve abstention behavior and response grounding in the evaluated domain-specific RAG setting.However, the framework's generalizability to other domains requires further empirical validation.