SteadyRoute: A Multi-Signal Router for Adaptive Retrieval-Augmented Generation, and Why Its Overhead Outweighed Its Savings

Adaptive retrieval-augmented generation (RAG) de-cides, question by question, whether to answer from the model’sown knowledge, retrieve once, or retrieve iteratively. Mostrouters make this choice from the question text alone. We builtSteadyRoute, a LightGBM router that also reads the model’sconfidence in a draft answer, the shape of the retrieval scores,and a short check of whether the retrieved evidence is sufficient.We trained it with paraphrased copies of each question so thatreworded questions would receive the same route. We evaluatedit on 600 HotpotQA and SQuAD questions with Llama 3.1 8B,keeping full accounting of every token the router itself consumes.SteadyRoute chose the oracle route for 80.5% of answerable testquestions, compared with 69.5% for a question-text router and68.8% for the majority route. The improvement did not translateinto savings. Collecting the extra signals took about two LLMcalls per question, so SteadyRoute used 28% more tokens thanalways retrieving once, with F1 of 0.688 against 0.720. Paraphrasetraining also failed to reduce decision flips (17.7% against 13.6%for the text router). Two findings explain this outcome. First, thecheapest adequate route itself changed for 34.7% of rewordings,so a flip is often correct behaviour rather than a defect. Second,a router pays for itself only if the signals it reads are alreadypart of the answering pipeline. We report these negative resultsin full so that others can check and build on them.