Recursive rejection sampling achieves at least three quarters of the optimal acceptance in multi-draft speculative decoding with independent drafts

Speculative decoding speeds up a large language model: a small draft model proposes tokens, and the large model verifies them without changing its output distribution. With k independent drafts per position, SpecInfer's recursive rejection sampling (RRS) tests drafts one at a time, with no guarantee we know of beyond 1/k. We prove that RRS achieves at least a fraction rho_k = 1 - max_{0<=a<=1}(a^k - a^{2k-1}) of the optimal acceptance, and that this is tight: rho_2 = 23/27, and rho_k decreases to 3/4. The proof writes RRS as one scalar recursion and lumps the tokens into two groups by the sign of the final residual, which can only lower RRS's acceptance and only raise the optimum. We also give KEY, an exact optimal verifier: after one sort and an interval-placement step, each draft draws a random key, and the draft with the smallest key is verified. On two small model pairs (temperature 1, four drafts), the mean acceptance gap of RRS is only 0.033, yet RRS rejects 1.35 times as often as the optimal verifier (pooled). The record includes the paper, its LaTeX source, and the code and certificates that reproduce every computed result.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-06
DOI
https://doi.org/10.5281/zenodo.24023724
Primary Topic
Natural Language Processing Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Recursive rejection sampling achieves at least three quarters of the optimal acceptance in multi-draft speculative decoding with independent drafts

Kundan Baliga, Aditya Byju, Aiden Xie
Zenodo (CERN European Organization for Nuclear Research)
Natural Language Processing Techniques
preprint

Recursive rejection sampling achieves at least three quarters of the optimal acceptance in multi-draft speculative decoding with independent drafts

Kundan Baliga, Aditya Byju, Aiden Xie
preprint en

Abstract

Speculative decoding speeds up a large language model: a small draft model proposes tokens, and the large model verifies them without changing its output distribution. With k independent drafts per position, SpecInfer's recursive rejection sampling (RRS) tests drafts one at a time, with no guarantee we know of beyond 1/k. We prove that RRS achieves at least a fraction rho_k = 1 - max_{0<=a<=1}(a^k - a^{2k-1}) of the optimal acceptance, and that this is tight: rho_2 = 23/27, and rho_k decreases to 3/4. The proof writes RRS as one scalar recursion and lumps the tokens into two groups by the sign of the final residual, which can only lower RRS's acceptance and only raise the optimum. We also give KEY, an exact optimal verifier: after one sort and an interval-placement step, each draft draws a random key, and the draft with the smallest key is verified. On two small model pairs (temperature 1, four drafts), the mean acceptance gap of RRS is only 0.033, yet RRS rejects 1.35 times as often as the optimal verifier (pooled). The record includes the paper, its LaTeX source, and the code and certificates that reproduce every computed result.

Zenodo (CERN European Organization for Nuclear Research)
Georgia Institute of Technology (US), University of Illinois Urbana-Champaign (US)
Natural Language Processing Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon