Don't Answer Twice: Diffusion Language Models That Only Generate What Changed

Language model requests often contain near-duplicates of earlier prompts. While autoregressive models must regenerate the entire response for each request, masked diffusion language models can place a previous answer on their canvas and only regenerate the segments changed by the new input. We investigate the conditions under which this method succeeds and the specific requirements for its implementation. Reuse is not free. Naive reuse fails due to length exposure bias, which occurs because training provides spans where the number of masks always equals the content length, leading the model to treat a slot's length as a factual constraint. LLaDA-8B restores between 94-100% of its correct numbers when using a slot of the correct length, but this drops to 0-47% if the slot is one token longer (compared to 0-36% for Dream-7B). Furthermore, a JSON scaffold with generous slots fails to recover any of 255 real fields correctly. To address this, we learned a NULL mode for 37 minutes on a laptop GPU which transforms the slot length from an equality into an upper bound, improving accuracy for real fields with two spare tokens from 16% to 93%. By using this mode, residual decoding solves 85.3% of 300 held-out near-duplicate math problems, which outperforms the 74.3% accuracy achieved by decoding from scratch (p < 0.001) and performs at a level similar to the cached answer provided in the prompt at 85.7%. This approach achieves these results using only an eighth of the forward passes and a tenth of the wall time. We also observe that residual decoding is more robust than the prompt itself, as an irrelevant clause causes a 13.1 point drop in the prompt's performance but only a 2.6 point drop for residual decoding (83.7% vs 73.9%, p = 0.028). A learned conflict head allows reuse for reworded requests in one additional pass, achieving 91.5% accuracy compared to 79.7% for scratch and 82.4% for the prompt (both p ≤ 0.004) while using 0.24x the passes of scratch. When testing on 1,795 real GSM-Plus perturbations, verifying the canvas answer once under the prompt achieves 77.7% compared to 74.4% for the prompt alone (p < 0.0001) at 0.578x the prompt's passes. In free-form chat, reuse reaches the same performance as scratch at 0.31x the passes but provides no advantage over a plain semantic cache. All targets were fixed before testing on sets that no system had previously decoded, and every miss is reported.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-15
DOI
https://doi.org/10.5281/zenodo.24253708
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Don't Answer Twice: Diffusion Language Models That Only Generate What Changed

Muhammad Umar Shaukat
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

Don't Answer Twice: Diffusion Language Models That Only Generate What Changed

Muhammad Umar Shaukat
preprint en

Abstract

Language model requests often contain near-duplicates of earlier prompts. While autoregressive models must regenerate the entire response for each request, masked diffusion language models can place a previous answer on their canvas and only regenerate the segments changed by the new input. We investigate the conditions under which this method succeeds and the specific requirements for its implementation. Reuse is not free. Naive reuse fails due to length exposure bias, which occurs because training provides spans where the number of masks always equals the content length, leading the model to treat a slot's length as a factual constraint. LLaDA-8B restores between 94-100% of its correct numbers when using a slot of the correct length, but this drops to 0-47% if the slot is one token longer (compared to 0-36% for Dream-7B). Furthermore, a JSON scaffold with generous slots fails to recover any of 255 real fields correctly. To address this, we learned a NULL mode for 37 minutes on a laptop GPU which transforms the slot length from an equality into an upper bound, improving accuracy for real fields with two spare tokens from 16% to 93%. By using this mode, residual decoding solves 85.3% of 300 held-out near-duplicate math problems, which outperforms the 74.3% accuracy achieved by decoding from scratch (p < 0.001) and performs at a level similar to the cached answer provided in the prompt at 85.7%. This approach achieves these results using only an eighth of the forward passes and a tenth of the wall time. We also observe that residual decoding is more robust than the prompt itself, as an irrelevant clause causes a 13.1 point drop in the prompt's performance but only a 2.6 point drop for residual decoding (83.7% vs 73.9%, p = 0.028). A learned conflict head allows reuse for reworded requests in one additional pass, achieving 91.5% accuracy compared to 79.7% for scratch and 82.4% for the prompt (both p ≤ 0.004) while using 0.24x the passes of scratch. When testing on 1,795 real GSM-Plus perturbations, verifying the canvas answer once under the prompt achieves 77.7% compared to 74.4% for the prompt alone (p < 0.0001) at 0.578x the prompt's passes. In free-form chat, reuse reaches the same performance as scratch at 0.31x the passes but provides no advantage over a plain semantic cache. All targets were fixed before testing on sets that no system had previously decoded, and every miss is reported.

Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
Don't Answer Twice: Diffusion Language Models That Only Generate What Changed — Muhammad Umar Shaukat · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS