When Sources Disagree: Accuracy-Preserving Batch Verification and the Limits of Source-Age Screening in Retrieval-Augmented Generation
Abstract Retrieval-augmented generation (RAG) can attach citations that contradict a generated claim or no longer reflect current facts. We revisit an experimentally evaluated citation-verification design using SciFact for semantic contradiction and a small, independently reviewed FreshQA case study for temporal change. On 200 balanced SciFact SUPPORT/CONTRADICT cases, a cross-encoder MiniLM-NLI baseline achieved 72.0% accuracy and 0.7144 macro-F1. The originally proposed negation-first shortcut reduced accuracy to 66.5%, despite reducing model invocations; consequently, we do not recommend that shortcut as an accuracy-preserving gate. In a separate five-repeat CPU throughput benchmark, length-sorted batch inference (batch size four) reduced total processing time from 51.779 s to 46.317 s (10.55%) while reproducing 100% of baseline predictions. Historical-source review identified four genuine FreshQA answer changes and one unchanged control. Applied to these five 148-day-old sources, the fixed 90-day age rule flagged all five: it alerted on all four corroborated changes and also on the one unchanged control (four true alerts, one false alert). Source age therefore indicates a need for re-verification, not proof of factual obsolescence. The findings support accuracy-preserving throughput optimization and motivate content-aware temporal checks; neither online latency improvement nor generalizable staleness classification is established.
Authors
- Michael Nthabiseng Moeti (ORCID: https://orcid.org/0000-0003-3783-7130)
- Kholofelo JONASE KGATLA (ORCID: https://orcid.org/0000-0001-6067-4929)
Institutions
- Tshwane University of Technology (ZA)
Publication Details
- Journal
- Research Square
- Published
- 2026-11-10
- DOI
- https://doi.org/10.21203/rs.3.rs-11678157/v1
- Primary Topic
- Topic Modeling
- Type
- preprint