BlinkThink: Training-Free Calibrated Dual-Process Routing for Discrete LLM Decisions

Discrete categorical choices -- including classification, multiple-choice QA, assertion verification, and guardrails -- constitute a substantial share of large language model (LLM) workloads. Routing every query through autoregressive Chain-of-Thought (CoT) reasoning incurs high latency, whereas zero-token prefill logit extraction (System One decision models) suffers from option-order bias and miscalibration. We present BlinkThink, a training-free, calibrated dual-process routing framework that couples a cyclic-permutation prefill evaluator (System 1) with selective CoT deliberation (System 2) via validation-tuned confidence gating. System 1 evaluates N cyclic option permutations in a single batched prefill pass (0 decoded tokens), applies bounded temperature scaling T*, and quantifies uncertainty via predictive margins, consensus, and total variation drift. Across google/gemma-4-12B-it and Qwen/Qwen3.5-4B on MMLU and BoolQ: (1) on gemma-4-12B-it MMLU, BlinkThink retains 99.4% of CoT accuracy (78.9% vs. 79.4%) with a 66.8% latency reduction (57.80 s to 19.16 s, 28.9% escalation) and 39.2% lower Expected Calibration Error (ECE) than CoT; (2) on Qwen3.5-4B, complete cyclic rotation (N = K = 4) neutralizes severe positional bias on MMLU (tau* = 0.00), reaching 68.4% in 190.4 ms (+24.7% over CoT's 43.7%), while on BoolQ transposed shifting (N = 2) reaches 86.1% in 149.1 ms (1,204.8 ms and 97.3% latency reduction under balanced routing); and (3) confidence ablations show pure Predictive Margin outperforms Max Softmax Probability for failure routing (AUROC = 0.6562 vs. 0.5737 on MMLU), whereas cross-shift drift reflects order sensitivity rather than epistemic error.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-20
DOI
https://doi.org/10.5281/zenodo.24406871
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

BlinkThink: Training-Free Calibrated Dual-Process Routing for Discrete LLM Decisions

Gábor Petneházi
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

BlinkThink: Training-Free Calibrated Dual-Process Routing for Discrete LLM Decisions

Gábor Petneházi
preprint en

Abstract

Discrete categorical choices -- including classification, multiple-choice QA, assertion verification, and guardrails -- constitute a substantial share of large language model (LLM) workloads. Routing every query through autoregressive Chain-of-Thought (CoT) reasoning incurs high latency, whereas zero-token prefill logit extraction (System One decision models) suffers from option-order bias and miscalibration. We present BlinkThink, a training-free, calibrated dual-process routing framework that couples a cyclic-permutation prefill evaluator (System 1) with selective CoT deliberation (System 2) via validation-tuned confidence gating. System 1 evaluates N cyclic option permutations in a single batched prefill pass (0 decoded tokens), applies bounded temperature scaling T*, and quantifies uncertainty via predictive margins, consensus, and total variation drift. Across google/gemma-4-12B-it and Qwen/Qwen3.5-4B on MMLU and BoolQ: (1) on gemma-4-12B-it MMLU, BlinkThink retains 99.4% of CoT accuracy (78.9% vs. 79.4%) with a 66.8% latency reduction (57.80 s to 19.16 s, 28.9% escalation) and 39.2% lower Expected Calibration Error (ECE) than CoT; (2) on Qwen3.5-4B, complete cyclic rotation (N = K = 4) neutralizes severe positional bias on MMLU (tau* = 0.00), reaching 68.4% in 190.4 ms (+24.7% over CoT's 43.7%), while on BoolQ transposed shifting (N = 2) reaches 86.1% in 149.1 ms (1,204.8 ms and 97.3% latency reduction under balanced routing); and (3) confidence ablations show pure Predictive Margin outperforms Max Softmax Probability for failure routing (AUROC = 0.6562 vs. 0.5737 on MMLU), whereas cross-shift drift reflects order sensitivity rather than epistemic error.

Zenodo (CERN European Organization for Nuclear Research)
University of Debrecen (HU)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon