An Auditable Binary Evidence-Escalation Protocol for Prompt-Constrained Retrieval-Augmented Question Answering
Technical-support and engineering knowledge services must decide whether one selected passage is sufficient or additional retrieved evidence should enter the generation prompt. Aggregate accuracy and mean context size conceal whether expansion recovers omitted evidence, adds no value, or harms an otherwise correct answer. We present an auditable binary evidence-escalation protocol for this operational decision. A logistic controller uses 27 pre-generation question and evidence-scoring features to select either one document or a model–task-specific full-candidate action; its outcome-derived development target is configuration-specific, and the resulting policy comparison is conditional on the single development-selected fit rather than an out-of-sample estimate of controller-selection performance. We evaluate deterministic distractor-augmented stress tests derived from SQuAD, WikiQA, and HotpotQA using two open-weight generators and one opaque GPT-compatible API. In the revision-locked GPT/API analysis, the SQuAD-Hard policy had the same observed accuracy as the full-candidate reference (0.9025; paired difference 0.0000, 95% bootstrap CI [−0.0175,0.0175]) while reducing mean prompted documents from 5.00 to 2.74 and reconstructed normalized prompt tokens by 40.8% (95% bootstrap CI [37.3%,44.4%]). No equivalence margin was prespecified. Under the present six-document candidate configuration, WikiQA-Hard provides no practical prompt-evidence benefit: the controller escalated 95.9% of examples and reduced mean prompted documents by only 0.21 of 6 (2.9% token reduction), whereas HotpotQA-Hard incurred an observed accuracy loss of 0.0350. These held-out comparisons quantify test-instance variation conditional on the selected controller; they do not include retraining or controller-selection uncertainty. Paired transition accounting and a bounded case audit distinguish selection, generation, and measurement failures. The protocol provides a design-time audit for prompt-evidence budgets in engineering knowledge and technical-support QA; these benchmark results do not establish production readiness, superiority to adaptive RAG, or latency, energy, or monetary savings.
Authors
- Yi Cao (ORCID: https://orcid.org/0000-0002-4222-8439)
- Jinyi Zhu
- Hao Wu
Institutions
- Wuxi University (CN)
Publication Details
- Journal
- Electronics
- Published
- 2026-10-15
- DOI
- https://doi.org/10.3390/electronics15204759
- Primary Topic
- Topic Modeling
- Type
- article
- Field-Weighted Citation Impact
- 0.00