When Sources Disagree: Accuracy-Preserving Batch Verification and the Limits of Source-Age Screening in Retrieval-Augmented Generation

Abstract Retrieval-augmented generation (RAG) can attach citations that contradict a generated claim or no longer reflect current facts. We revisit an experimentally evaluated citation-verification design using SciFact for semantic contradiction and a small, independently reviewed FreshQA case study for temporal change. On 200 balanced SciFact SUPPORT/CONTRADICT cases, a cross-encoder MiniLM-NLI baseline achieved 72.0% accuracy and 0.7144 macro-F1. The originally proposed negation-first shortcut reduced accuracy to 66.5%, despite reducing model invocations; consequently, we do not recommend that shortcut as an accuracy-preserving gate. In a separate five-repeat CPU throughput benchmark, length-sorted batch inference (batch size four) reduced total processing time from 51.779 s to 46.317 s (10.55%) while reproducing 100% of baseline predictions. Historical-source review identified four genuine FreshQA answer changes and one unchanged control. Applied to these five 148-day-old sources, the fixed 90-day age rule flagged all five: it alerted on all four corroborated changes and also on the one unchanged control (four true alerts, one false alert). Source age therefore indicates a need for re-verification, not proof of factual obsolescence. The findings support accuracy-preserving throughput optimization and motivate content-aware temporal checks; neither online latency improvement nor generalizable staleness classification is established.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-10
DOI
https://doi.org/10.21203/rs.3.rs-11678157/v1
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

When Sources Disagree: Accuracy-Preserving Batch Verification and the Limits of Source-Age Screening in Retrieval-Augmented Generation

Michael Nthabiseng Moeti, Kholofelo JONASE KGATLA
Research Square
Topic Modeling
preprint

When Sources Disagree: Accuracy-Preserving Batch Verification and the Limits of Source-Age Screening in Retrieval-Augmented Generation

Michael Nthabiseng Moeti, Kholofelo JONASE KGATLA
preprint en

Abstract

Abstract Retrieval-augmented generation (RAG) can attach citations that contradict a generated claim or no longer reflect current facts. We revisit an experimentally evaluated citation-verification design using SciFact for semantic contradiction and a small, independently reviewed FreshQA case study for temporal change. On 200 balanced SciFact SUPPORT/CONTRADICT cases, a cross-encoder MiniLM-NLI baseline achieved 72.0% accuracy and 0.7144 macro-F1. The originally proposed negation-first shortcut reduced accuracy to 66.5%, despite reducing model invocations; consequently, we do not recommend that shortcut as an accuracy-preserving gate. In a separate five-repeat CPU throughput benchmark, length-sorted batch inference (batch size four) reduced total processing time from 51.779 s to 46.317 s (10.55%) while reproducing 100% of baseline predictions. Historical-source review identified four genuine FreshQA answer changes and one unchanged control. Applied to these five 148-day-old sources, the fixed 90-day age rule flagged all five: it alerted on all four corroborated changes and also on the one unchanged control (four true alerts, one false alert). Source age therefore indicates a need for re-verification, not proof of factual obsolescence. The findings support accuracy-preserving throughput optimization and motivate content-aware temporal checks; neither online latency improvement nor generalizable staleness classification is established.

Research Square
Tshwane University of Technology (ZA)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon