Nano-jev: Calibrated Cross-Encoder Micro-Decisions for Retrieval-Augmented Generation with Confidence Cascading

Retrieval-Augmented Generation (RAG) pipelines repeatedly make small judgments, such as whether a passage is relevant, whether the retrieved context is sufficient and whether an answer is grounded. These are usually delegated to the generating large language model (LLM), at notable latency and cost, or to score thresholds without a probabilistic interpretation. We study nano-jev, a 33.4\,M-parameter cross-encoder that treats relevance (3-way), sufficiency and groundedness (binary) as typed, closed-set classification problems and returns temperature-calibrated probabilities. Each (question, option, context) triple is scored by a single-logit head, which makes predictions invariant to option order. On held-out halves of the HotpotQA, SQuAD~2.0 and MultiNLI validation sets, nano-jev reaches mean accuracies of 0.812 (relevance), 0.843 (sufficiency) and 0.841 (groundedness) over five seeds, comparable to a 1-pass classifier trained on the same data, which is 2.5$\times$ faster. Dedicated checkers such as MiniCheck remain stronger on groundedness (0.866). nano-jev runs 55-93$\times$ faster than a prompted 4\,B-parameter LLM. On unseen datasets (MuSiQue, VitaminC), accuracy drops by 15-20 points and ECE rises from about 0.03 to about 0.17, so target-domain re-calibration is advisable. A confidence cascade that escalates uncertain decisions to the LLM improves in-distribution groundedness (0.868 vs. 0.841, $p=0.02$) with about a quarter of decisions escalated, but trades accuracy for latency out of distribution. In a small end-to-end test, nano-jev relevance filtering improves HotpotQA answer F1 by 2.1 points at about 39ms per question. Code and weights are released as an open-source package.

Authors

Institutions

Publication Details

Journal
ICCK Transactions on Intelligent Systematics
Published
2026-11-16
DOI
https://doi.org/10.62762/tis.2026.197904
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Nano-jev: Calibrated Cross-Encoder Micro-Decisions for Retrieval-Augmented Generation with Confidence Cascading

Subham Divakar, Rojalina Priyadarshini
3 links
ICCK Transactions on Intelligent Systematics
Topic Modeling
article

Nano-jev: Calibrated Cross-Encoder Micro-Decisions for Retrieval-Augmented Generation with Confidence Cascading

Subham Divakar, Rojalina Priyadarshini
article en

Abstract

Retrieval-Augmented Generation (RAG) pipelines repeatedly make small judgments, such as whether a passage is relevant, whether the retrieved context is sufficient and whether an answer is grounded. These are usually delegated to the generating large language model (LLM), at notable latency and cost, or to score thresholds without a probabilistic interpretation. We study nano-jev, a 33.4\,M-parameter cross-encoder that treats relevance (3-way), sufficiency and groundedness (binary) as typed, closed-set classification problems and returns temperature-calibrated probabilities. Each (question, option, context) triple is scored by a single-logit head, which makes predictions invariant to option order. On held-out halves of the HotpotQA, SQuAD~2.0 and MultiNLI validation sets, nano-jev reaches mean accuracies of 0.812 (relevance), 0.843 (sufficiency) and 0.841 (groundedness) over five seeds, comparable to a 1-pass classifier trained on the same data, which is 2.5$\times$ faster. Dedicated checkers such as MiniCheck remain stronger on groundedness (0.866). nano-jev runs 55-93$\times$ faster than a prompted 4\,B-parameter LLM. On unseen datasets (MuSiQue, VitaminC), accuracy drops by 15-20 points and ECE rises from about 0.03 to about 0.17, so target-domain re-calibration is advisable. A confidence cascade that escalates uncertain decisions to the LLM improves in-distribution groundedness (0.868 vs. 0.841, $p=0.02$) with about a quarter of decisions escalated, but trades accuracy for latency out of distribution. In a small end-to-end test, nano-jev relevance filtering improves HotpotQA answer F1 by 2.1 points at about 39ms per question. Code and weights are released as an open-source package.

ICCK Transactions on Intelligent SystematicsVol. 3(4)
C.V. Raman Global University (IN)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon