Megadose The AI curve, compressed.

LoopFill: Pipelined Multi-Token Speculative Decoding for Looped Transformers

· ArXiv · AI/CL/LG ·
LoopFill reports average decoding speedups of 5.85× on Ouro-2.6B and 5.13× on Huginn-3.5B across seven benchmarks.

Looped Transformers reuse the same layers, but repeated passes make token generation slow. The training-free method combines multi-token suffix drafts with shallow-loop predictions in one draft tree, allowing several future tokens to enter the pipeline together. Full-depth verification preserves the model’s autoregressive output distribution. The reported speedups exceed those of LoopAhead, the strongest baseline tested, by 40% and 25% on the two models, respectively.

ArXiv · AI/CL/LG's note

score 5

Categories: Research