FT-Behavioural Router: Knowledge vs Behaviour Separation in Adaptive Hybrid Inference
Hybrid assistants combine retrieval-augmented generation (RAG), tools, and multiple model sizes, but calling a large LLM with retrieval on every query is costly. We study a fine-tuned (FT) behavioural router that selects among four actions: answer as a small model, retrieve and answer as the same small model (FT+RAG), invoke tools, or escalate to a large LLM, while keeping factual knowledge non-parametric. We do not claim inventing adaptive RAG or LLM cascading. On in-domain workplace eval_v2 (n=133), FT DistilBERT reaches accuracy 0.9699 and MiniLM 0.8872, above a prompt-rubric simulation (0.7744; not an LLM API). Exact McNemar and paired bootstrap tests on the same eval items find DistilBERT significantly above MiniLM and keyword (measured p<0.01). On a hard OOD split (n=104; never used for training) DistilBERT drops to 0.8750 and the rubric collapses to 0.5000. A remapped public eval (SQuAD 1.1, HotpotQA, CLINC-150; n=240) is a second domain: workplace cue rules fall to about 0.25–0.28; workplace-trained DistilBERT transfers at 0.5000 while remapped-train DistilBERT reaches 0.9250. A Phase 2 unit-cost simulation shows FT mean cost roughly 0.25× always-RAG+escalate; this is not a cloud bill and no live answer EM/F1 is reported. We deepen the related-work contrast, error analysis, limitations, and when-FT-is-worth-it criteria.
Authors
- Vijay Kumar
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-09-25
- DOI
- https://doi.org/10.5281/zenodo.22968232
- Primary Topic
- Topic Modeling
- Type
- preprint