Metacognitive Decoding: Efficient Inference for Diffusion Language Agents
The authors report 20.6% fewer inference FLOPs on average across five benchmarks, with competitive task accuracy.
Their lightweight controller plugs into a diffusion language model and learns when to finalize tokens and how far to extend a response. It is trained on agent trajectories to balance correctness against generation cost. The reported savings are measured against the most efficient baseline tested. Ablation studies also support the controller’s effectiveness, according to the abstract.
ArXiv · AI/CL/LG's note
Their lightweight controller plugs into a diffusion language model and learns when to finalize tokens and how far to extend a response. It is trained on agent trajectories to balance correctness against generation cost. The reported savings are measured against the most efficient baseline tested. Ablation studies also support the controller’s effectiveness, according to the abstract.
ArXiv · AI/CL/LG's note
score 5