BlinkThink: Training-Free Calibrated Dual-Process Routing for Discrete LLM Decisions
Discrete categorical choices -- including classification, multiple-choice QA, assertion verification, and guardrails -- constitute a substantial share of large language model (LLM) workloads. Routing every query through autoregressive Chain-of-Thought (CoT) reasoning incurs high latency, whereas zero-token prefill logit extraction (System One decision models) suffers from option-order bias and miscalibration. We present BlinkThink, a training-free, calibrated dual-process routing framework that couples a cyclic-permutation prefill evaluator (System 1) with selective CoT deliberation (System 2) via validation-tuned confidence gating. System 1 evaluates N cyclic option permutations in a single batched prefill pass (0 decoded tokens), applies bounded temperature scaling T*, and quantifies uncertainty via predictive margins, consensus, and total variation drift. Across google/gemma-4-12B-it and Qwen/Qwen3.5-4B on MMLU and BoolQ: (1) on gemma-4-12B-it MMLU, BlinkThink retains 99.4% of CoT accuracy (78.9% vs. 79.4%) with a 66.8% latency reduction (57.80 s to 19.16 s, 28.9% escalation) and 39.2% lower Expected Calibration Error (ECE) than CoT; (2) on Qwen3.5-4B, complete cyclic rotation (N = K = 4) neutralizes severe positional bias on MMLU (tau* = 0.00), reaching 68.4% in 190.4 ms (+24.7% over CoT's 43.7%), while on BoolQ transposed shifting (N = 2) reaches 86.1% in 149.1 ms (1,204.8 ms and 97.3% latency reduction under balanced routing); and (3) confidence ablations show pure Predictive Margin outperforms Max Softmax Probability for failure routing (AUROC = 0.6562 vs. 0.5737 on MMLU), whereas cross-shift drift reflects order sensitivity rather than epistemic error.
Authors
- Gábor Petneházi (ORCID: https://orcid.org/0000-0001-6579-4583)
Institutions
- University of Debrecen (HU)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-20
- DOI
- https://doi.org/10.5281/zenodo.24406871
- Primary Topic
- Topic Modeling
- Type
- preprint