Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs
Flash-dLLM targets the memory traffic bottleneck that appears when diffusion LLMs combine KV-cache reuse with parallel token verification.
The paper presents a training-free inference framework for diffusion LLMs, using an I/O-aware fused KV-cache kernel to cut redundant GPU memory movement. It also adds a KV-cache-driven draft-and-verify decoding method where the same dLLM acts as both drafter and verifier. The authors report preserved generation quality with better scaling to longer sequences and larger batches. In experiments, Flash-dLLM beats Elastic-Cache by 5.1x on GSM8K and 11.0x on HumanEval. ArXiv · AI/CL/LG's note
The paper presents a training-free inference framework for diffusion LLMs, using an I/O-aware fused KV-cache kernel to cut redundant GPU memory movement. It also adds a KV-cache-driven draft-and-verify decoding method where the same dLLM acts as both drafter and verifier. The authors report preserved generation quality with better scaling to longer sequences and larger batches. In experiments, Flash-dLLM beats Elastic-Cache by 5.1x on GSM8K and 11.0x on HumanEval. ArXiv · AI/CL/LG's note
score 5