Adaptive Retrieval and Evidence Calibration for Reliable Retrieval-Augmented Generation

Retrieval-augmented generation (RAG) can expose a response to incorrect evidence when a retrieved source is plausible but mismatched. This paper presents ACRE, a compact pipeline that couples query-dependent reranking depth, source-evidence confidence estimation, and a selective output decision. A BM25 candidate list is augmented with lexical similarity signals; logistic confidence models select one, three, or five candidates and determine whether a verbatim, source-attributed answer should be issued. We evaluate on 300 authentic news reports and 50 out-of-index reports from the Lee text corpora, using 1,400 reproducibly derived source-identification queries. On the 280-query held-out split, adaptive retrieval matches the BM25 top-1 source accuracy of 77.86% while requiring only 1.73 candidates per query in the assigned reranking depth, versus five for the fixed-depth alternative. With the development-selected abstention threshold, the system responds to 80.36% of test queries at 91.11% accepted-source accuracy; its expected calibration error is 0.039. The results demonstrate an economical approach to coupling retrieval efficiency with calibrated source attribution in an explicitly extractive RAG setting.

Authors

Institutions

Publication Details

Journal
Jisuanji shenghuojia.
Published
2026-11-13
DOI
https://doi.org/10.54097/arjehm36
Primary Topic
Information Retrieval and Search Behavior
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Adaptive Retrieval and Evidence Calibration for Reliable Retrieval-Augmented Generation

Clara N. Whitmore
3 links
Jisuanji shenghuojia.
Information Retrieval and Search Behavior
article

Adaptive Retrieval and Evidence Calibration for Reliable Retrieval-Augmented Generation

Clara N. Whitmore
article en

Abstract

Retrieval-augmented generation (RAG) can expose a response to incorrect evidence when a retrieved source is plausible but mismatched. This paper presents ACRE, a compact pipeline that couples query-dependent reranking depth, source-evidence confidence estimation, and a selective output decision. A BM25 candidate list is augmented with lexical similarity signals; logistic confidence models select one, three, or five candidates and determine whether a verbatim, source-attributed answer should be issued. We evaluate on 300 authentic news reports and 50 out-of-index reports from the Lee text corpora, using 1,400 reproducibly derived source-identification queries. On the 280-query held-out split, adaptive retrieval matches the BM25 top-1 source accuracy of 77.86% while requiring only 1.73 candidates per query in the assigned reranking depth, versus five for the fixed-depth alternative. With the development-selected abstention threshold, the system responds to 80.36% of test queries at 91.11% accepted-source accuracy; its expected calibration error is 0.039. The results demonstrate an economical approach to coupling retrieval efficiency with calibrated source attribution in an explicitly extractive RAG setting.

Jisuanji shenghuojia.Vol. 15(2)
University of Alberta (CA)
Information Retrieval and Search Behavior
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon