Fused Correction: Deleting the Correction Forward in Pre-Check-Gated Greedy Retrieval Decoding
Pre-check-gated greedy retrieval loops pay two target passes per accepted step: one 6-row verify and one 1-row correction forward that materializes the bonus token's logits for the next d0 pre-check. We show the correction forward can be deleted exactly in this gated regime: after a full-accept step the bonus id is known but its KV row was never written, so the next step writes the bonus as feed[0] of a single 6-row sweep plus five history drafts and post-judges all rows. On locked repetition we measure 1.43x/1.52x paired (Llama-3.2-3B, ids-identical; 1.77x AABB remeasure since the same-binary --no-fuse gate) and 1.11-1.32x (Qwen2.5-0.5B, three same-window pairs; neutral 1.04x --gpu off remeasure); repeat-penalty decoding (1.1) preserves parity and the gain (1.59x/2.03x paired, window-caveated, capped at the 1.71x sweep-ideal). On non-repetitive prompts the construction never chains and output is unchanged (0.96-1.03x neutral); mixed-partial chaining loses (0.74x measured). Standard no-pre-check loops (HF transformers assisted generation, vLLM spec decode, llama.cpp lookup) already carry the bonus into the next pass and never pay this gate. Measured on Aegean-TGE (native Windows C++20 GGUF inference engine, Qwen2/Llama, CPU with opt-in Vulkan offload), commits 99a92c6 + 269b923, tge_tests 141/141 green. Scope: greedy, retrieval drafting, CPU batch-1.
Authors
- Yunus Emre Zahiroğlu
Institutions
- Van Yüzüncü Yıl Üniversitesi (TR)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-05
- DOI
- https://doi.org/10.5281/zenodo.23990528
- Primary Topic
- Natural Language Processing Techniques
- Type
- preprint