On the SPOT: Enhancing Speculative Decoding for Diffusion Language Models via Commit-Order Prediction
SPOT delivers up to 3.4× the throughput of standard decoding on SDAR-8B while preserving exact decoding.
The authors identify token order as the bottleneck: predictions from the previous pass already contain the right token at over 90% of positions, but do not identify which position comes next. Their 620,000-parameter drafter predicts that order and proposes candidate continuations for verification in a single target-model pass. Sharing Gumbel noise between drafter and target keeps verification exact at any temperature. A separate variant, SPOT-Sets, drafts groups of positions and tokens, improving throughput by 59% on average over dynamic threshold decoding alone.
HF Daily Papers' note
The authors identify token order as the bottleneck: predictions from the previous pass already contain the right token at over 90% of positions, but do not identify which position comes next. Their 620,000-parameter drafter predicts that order and proposes candidate continuations for verification in a single target-model pass. Sharing Gumbel noise between drafter and target keeps verification exact at any temperature. A separate variant, SPOT-Sets, drafts groups of positions and tokens, improving throughput by 59% on average over dynamic threshold decoding alone.
HF Daily Papers' note
score 5