When is LLM-based reasoning retrieval worth the cost? Lightweight cost-aware query routing for retrieval-augmented generation

Retrieval-Augmented Generation (RAG) systems face a cost–effectiveness trade-off: inexpensive retrievers are efficient, whereas reasoning-based retrievers can improve retrieval quality at substantially higher computational cost. We introduce CARE (Cost-aware Adaptive REtrieval), a lightweight cost-aware binary routing method that decides whether each query should be handled by an inexpensive Hybrid retriever or escalated to the ReSP (Retrieve, Summarize, Plan) reasoning-based retriever. The routing decision is made by a lightweight logistic-regression gate using query-level and inexpensive probe-retrieval features. The evaluation covers two BEIR question-answering (QA) datasets, HotpotQA and Natural Questions (NQ), comprising 7,914,797 documents and 10,857 benchmark queries in total, of which 4097 constitute the fully held-out test sets used for evaluation. Using dataset-specific thresholds selected independently on the corresponding validation splits ( τ = 0.30 for HotpotQA and τ = 0.60 for NQ), CARE achieves utilities of 0.8390 and 0.7596, respectively. These dataset-specific operating points yield a macro-average utility of 0.7993 and a macro-average expensive-branch activation rate of 49.31%. On HotpotQA, CARE approximately matches ReSP in retrieval effectiveness, although its Recall@100 is slightly lower (0.8770 versus 0.8795). Its 0.0007 utility advantage is therefore driven by the routing-cost term rather than by superior retrieval effectiveness, with expensive-branch activation reduced from 100% to 88.79%. On NQ, CARE provides a clearer cost-sensitive benefit, achieving 0.7596 utility with only 9.82% activation.

Authors

Institutions

Publication Details

Journal
Information Processing & Management
Published
2026-11-16
DOI
https://doi.org/10.1016/j.ipm.2026.105287
Citations
1
Primary Topic
Information Retrieval and Search Behavior
Type
article
Field-Weighted Citation Impact
2.57
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

When is LLM-based reasoning retrieval worth the cost? Lightweight cost-aware query routing for retrieval-augmented generation

Rafael Molina-Carmona, Alberto Real-Fernández, Sergio Arjona-Giner, Faraón Llorens Largo
1 citations
Information Processing & Management
Information Retrieval and Search Behavior
2.57
article

When is LLM-based reasoning retrieval worth the cost? Lightweight cost-aware query routing for retrieval-augmented generation

Rafael Molina-Carmona, Alberto Real-Fernández, Sergio Arjona-Giner, Faraón Llorens Largo
article en
1 citations

Abstract

Retrieval-Augmented Generation (RAG) systems face a cost–effectiveness trade-off: inexpensive retrievers are efficient, whereas reasoning-based retrievers can improve retrieval quality at substantially higher computational cost. We introduce CARE (Cost-aware Adaptive REtrieval), a lightweight cost-aware binary routing method that decides whether each query should be handled by an inexpensive Hybrid retriever or escalated to the ReSP (Retrieve, Summarize, Plan) reasoning-based retriever. The routing decision is made by a lightweight logistic-regression gate using query-level and inexpensive probe-retrieval features. The evaluation covers two BEIR question-answering (QA) datasets, HotpotQA and Natural Questions (NQ), comprising 7,914,797 documents and 10,857 benchmark queries in total, of which 4097 constitute the fully held-out test sets used for evaluation. Using dataset-specific thresholds selected independently on the corresponding validation splits ( τ = 0.30 for HotpotQA and τ = 0.60 for NQ), CARE achieves utilities of 0.8390 and 0.7596, respectively. These dataset-specific operating points yield a macro-average utility of 0.7993 and a macro-average expensive-branch activation rate of 49.31%. On HotpotQA, CARE approximately matches ReSP in retrieval effectiveness, although its Recall@100 is slightly lower (0.8770 versus 0.8795). Its 0.0007 utility advantage is therefore driven by the routing-cost term rather than by superior retrieval effectiveness, with expensive-branch activation reduced from 100% to 88.79%. On NQ, CARE provides a clearer cost-sensitive benefit, achieving 0.7596 utility with only 9.82% activation.

Information Processing & ManagementVol. 64(2)
University of Alicante (ES)
Openalex Percentile: Top 13%
Information Retrieval and Search Behavior
2.57
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon