LoopFill: Pipelined Multi-Token Speculative Decoding for Looped Transformers
LoopFill reports up to 5.85× average decoding speedup without additional training.
Across seven benchmarks, average speedups over greedy autoregressive decoding reached 5.85× on Ouro-2.6B and 5.13× on Huginn-3.5B. Those gains exceeded LoopAhead, the strongest baseline, by 40% and 25%, respectively. LoopFill combines multi-token suffix drafts with shallow-loop predictions in one draft tree, allowing several future tokens to enter the pipeline together. Full-depth verification preserves the model’s autoregressive output distribution.
HF Daily Papers' note
Across seven benchmarks, average speedups over greedy autoregressive decoding reached 5.85× on Ouro-2.6B and 5.13× on Huginn-3.5B. Those gains exceeded LoopAhead, the strongest baseline, by 40% and 25%, respectively. LoopFill combines multi-token suffix drafts with shallow-loop predictions in one draft tree, allowing several future tokens to enter the pipeline together. Full-depth verification preserves the model’s autoregressive output distribution.
HF Daily Papers' note
score 5