Rethink‐ RAG : Dynamic Retrieval Decision and Cooperative Verification for Reliable Multi‐Hop Question Answering

ABSTRACT Reliable multi‐hop question answering is important for knowledge‐intensive expert systems, where a wrong intermediate fact can propagate into an incorrect final decision. Existing Retrieval‐Augmented Generation (RAG) methods often retrieve for every sub‐question, even when the language model can answer a step from parametric knowledge, and this indiscriminate retrieval can introduce irrelevant or contradictory evidence. We propose Rethink‐RAG, a training‐free framework that combines dynamic retrieval decision with a cooperative answer verification pipeline. The dynamic retrieval decision module generates diverse rewrites of each sub‐question and applies majority voting to decide whether external retrieval is required. The cooperative verification pipeline then filters retrieved documents, quantifies support and opposition through token‐logit evidence, and arbitrates each candidate sub‐answer with an adaptive confidence threshold. Experiments on 2WikiMultihopQA, HotpotQA, and MuSiQueQA show that Rethink‐RAG improves answer reliability on knowledge‐intensive reasoning tasks, achieving 49.00% ACC on 2WikiMultihopQA and 49.60% ACC on HotpotQA with the Llama3‐8B evaluator. Additional revision experiments quantify the latency/API‐call trade‐off, sensitivity to the decision‐boundary factor, and the limitations observed on MuSiQueQA, giving a more complete view of the framework's practical operating range.