Megadose The AI curve, compressed.

LoopFill: Pipelined Multi-Token Speculative Decoding for Looped Transformers

· HF Daily Papers ·
LoopFill reports up to 5.85× average decoding speedup without additional training.

Across seven benchmarks, average speedups over greedy autoregressive decoding reached 5.85× on Ouro-2.6B and 5.13× on Huginn-3.5B. Those gains exceeded LoopAhead, the strongest baseline, by 40% and 25%, respectively. LoopFill combines multi-token suffix drafts with shallow-loop predictions in one draft tree, allowing several future tokens to enter the pipeline together. Full-depth verification preserves the model’s autoregressive output distribution.

HF Daily Papers' note

score 5

Categories: Research