REAL Loss: Optimizing Real Expected Acceptance Length for Speculative Decoding
REAL increased draft-token acceptance in a controlled comparison by changing only the training loss.
The paper identifies a gap in earlier speculative-decoding objectives that can send training gradients in the wrong direction. REAL uses path weights to estimate expected acceptance length without bias, plus a variance-reduction technique. With a Qwen3.6-35B-A3B target and Domino drafter, it outperformed WTV, VSD, DBLast and the static LK baseline across nine task subsets and five evaluation seeds. The comparison required no changes to the drafter architecture or inference procedure.
ArXiv · AI/CL/LG's note
The paper identifies a gap in earlier speculative-decoding objectives that can send training gradients in the wrong direction. REAL uses path weights to estimate expected acceptance length without bias, plus a variance-reduction technique. With a Qwen3.6-35B-A3B target and Domino drafter, it outperformed WTV, VSD, DBLast and the static LK baseline across nine task subsets and five evaluation seeds. The comparison required no changes to the drafter architecture or inference procedure.
ArXiv · AI/CL/LG's note
score 5