Choosing Evidence Before Answering: Single-Retrieval Decisions RAG versus Embedding-Only RAG

Embedding similarity identifies passages related to a question, but topical similarity does not establish answer support. We evaluate a fixed single-retrieval policy that scores embedding candidates against an explicit four-level support rubric using the OpenAI Decisions API, then selects evidence under a fixed threshold and token budget. The method is unchanged throughout the study. The main evaluation compares this policy with the application's embedding-only selection in paired runs on 600 newly sampled MuSiQue-Ans development questions. Both arms use the same native research worker and qualitative writer; only the Decisions feature gate differs. The sample excludes previously evaluated question IDs and exact normalized questions and targets a fixed 58/25/17 mixture of two-, three- and four-hop questions. The Decisions policy obtains mean automated, quote-extracted answer-content F1 46.66 versus 39.75 for embedding-only selection. The paired mean difference is +6.91 points, with nominal 95% hop-stratified bootstrap interval [+3.65, +10.27]. F1 measures normalized answer-token overlap. Higher F1 with 47.83% fewer selected passages (2.61 versus 5.00 per question) shows a favorable observed trade-off between answer overlap and selected evidence quantity. Annotated support recall is lower (60.82% versus 70.99%); factuality and citation entailment were not assessed. Post-hoc accounting of successfully metered API usage, including corpus and query embeddings, scoring, research and writing, estimates 38.03% higher cost for the same 600-question workload under the observed cache behaviour. A complementary, previously frozen 200-question HotpotQA study with a standardized short-answer reader finds an F1 gain of 8.30 points (nominal 95% paired interval [2.97, 13.72]). Equal-count and common-budget diagnostics support a useful ranking signal in that setting. These results concern evidence selection from supplied local paragraph pools, not full-corpus retrieval. Shared sources, public benchmark exposure, automated answer extraction, one model family and the absence of new human adjudication limit the claim. Conditional on the saved generations and stated resampling assumptions, the fresh comparison supports a positive answer-content F1 difference; unresolved source dependence limits population inference, and a gain exceeding the five-point working practical margin is not established. Working paper / preprint; not peer reviewed. The companion Source_and_Evidence.zip contains the frozen sample, protocols, saved outcomes and numerical reanalysis tools. It documents its reproduction boundaries and does not distribute the full private application or enable independent regeneration of native answers. Funding: no external research funding reported. Competing interests: none declared by the author.