When is LLM-based reasoning retrieval worth the cost? Lightweight cost-aware query routing for retrieval-augmented generation
Retrieval-Augmented Generation (RAG) systems face a cost–effectiveness trade-off: inexpensive retrievers are efficient, whereas reasoning-based retrievers can improve retrieval quality at substantially higher computational cost. We introduce CARE (Cost-aware Adaptive REtrieval), a lightweight cost-aware binary routing method that decides whether each query should be handled by an inexpensive Hybrid retriever or escalated to the ReSP (Retrieve, Summarize, Plan) reasoning-based retriever. The routing decision is made by a lightweight logistic-regression gate using query-level and inexpensive probe-retrieval features. The evaluation covers two BEIR question-answering (QA) datasets, HotpotQA and Natural Questions (NQ), comprising 7,914,797 documents and 10,857 benchmark queries in total, of which 4097 constitute the fully held-out test sets used for evaluation. Using dataset-specific thresholds selected independently on the corresponding validation splits ( τ = 0.30 for HotpotQA and τ = 0.60 for NQ), CARE achieves utilities of 0.8390 and 0.7596, respectively. These dataset-specific operating points yield a macro-average utility of 0.7993 and a macro-average expensive-branch activation rate of 49.31%. On HotpotQA, CARE approximately matches ReSP in retrieval effectiveness, although its Recall@100 is slightly lower (0.8770 versus 0.8795). Its 0.0007 utility advantage is therefore driven by the routing-cost term rather than by superior retrieval effectiveness, with expensive-branch activation reduced from 100% to 88.79%. On NQ, CARE provides a clearer cost-sensitive benefit, achieving 0.7596 utility with only 9.82% activation.
Authors
- Rafael Molina-Carmona (ORCID: https://orcid.org/0000-0001-5168-4500)
- Alberto Real-Fernández (ORCID: https://orcid.org/0000-0003-0940-6683)
- Sergio Arjona-Giner (ORCID: https://orcid.org/0009-0003-9182-9518)
- Faraón Llorens Largo (ORCID: https://orcid.org/0000-0002-2117-0784)
Institutions
- University of Alicante (ES)
Publication Details
- Journal
- Information Processing & Management
- Published
- 2026-11-16
- DOI
- https://doi.org/10.1016/j.ipm.2026.105287
- Citations
- 1
- Primary Topic
- Information Retrieval and Search Behavior
- Type
- article
- Field-Weighted Citation Impact
- 2.57