Fused Correction: Deleting the Correction Forward in Pre-Check-Gated Greedy Retrieval Decoding

Pre-check-gated greedy retrieval loops pay two target passes per accepted step: one 6-row verify and one 1-row correction forward that materializes the bonus token's logits for the next d0 pre-check. We show the correction forward can be deleted exactly in this gated regime: after a full-accept step the bonus id is known but its KV row was never written, so the next step writes the bonus as feed[0] of a single 6-row sweep plus five history drafts and post-judges all rows. On locked repetition we measure 1.43x/1.52x paired (Llama-3.2-3B, ids-identical; 1.77x AABB remeasure since the same-binary --no-fuse gate) and 1.11-1.32x (Qwen2.5-0.5B, three same-window pairs; neutral 1.04x --gpu off remeasure); repeat-penalty decoding (1.1) preserves parity and the gain (1.59x/2.03x paired, window-caveated, capped at the 1.71x sweep-ideal). On non-repetitive prompts the construction never chains and output is unchanged (0.96-1.03x neutral); mixed-partial chaining loses (0.74x measured). Standard no-pre-check loops (HF transformers assisted generation, vLLM spec decode, llama.cpp lookup) already carry the bonus into the next pass and never pay this gate. Measured on Aegean-TGE (native Windows C++20 GGUF inference engine, Qwen2/Llama, CPU with opt-in Vulkan offload), commits 99a92c6 + 269b923, tge_tests 141/141 green. Scope: greedy, retrieval drafting, CPU batch-1.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-05
DOI
https://doi.org/10.5281/zenodo.23990528
Primary Topic
Natural Language Processing Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Fused Correction: Deleting the Correction Forward in Pre-Check-Gated Greedy Retrieval Decoding

Yunus Emre Zahiroğlu
Zenodo (CERN European Organization for Nuclear Research)
Natural Language Processing Techniques
preprint

Fused Correction: Deleting the Correction Forward in Pre-Check-Gated Greedy Retrieval Decoding

Yunus Emre Zahiroğlu
preprint en

Abstract

Pre-check-gated greedy retrieval loops pay two target passes per accepted step: one 6-row verify and one 1-row correction forward that materializes the bonus token's logits for the next d0 pre-check. We show the correction forward can be deleted exactly in this gated regime: after a full-accept step the bonus id is known but its KV row was never written, so the next step writes the bonus as feed[0] of a single 6-row sweep plus five history drafts and post-judges all rows. On locked repetition we measure 1.43x/1.52x paired (Llama-3.2-3B, ids-identical; 1.77x AABB remeasure since the same-binary --no-fuse gate) and 1.11-1.32x (Qwen2.5-0.5B, three same-window pairs; neutral 1.04x --gpu off remeasure); repeat-penalty decoding (1.1) preserves parity and the gain (1.59x/2.03x paired, window-caveated, capped at the 1.71x sweep-ideal). On non-repetitive prompts the construction never chains and output is unchanged (0.96-1.03x neutral); mixed-partial chaining loses (0.74x measured). Standard no-pre-check loops (HF transformers assisted generation, vLLM spec decode, llama.cpp lookup) already carry the bonus into the next pass and never pay this gate. Measured on Aegean-TGE (native Windows C++20 GGUF inference engine, Qwen2/Llama, CPU with opt-in Vulkan offload), commits 99a92c6 + 269b923, tge_tests 141/141 green. Scope: greedy, retrieval drafting, CPU batch-1.

Zenodo (CERN European Organization for Nuclear Research)
Van Yüzüncü Yıl Üniversitesi (TR)
Natural Language Processing Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon