Towards Efficient Reasoning: Learning Causal Shortcuts for Diffusion Language Models
The paper says diffusion language models reason better when training highlights “causal shortcut” token chains through the answer.
The authors argue that random masking leaves DLMs searching a much larger space than autoregressive models, making key reasoning tokens harder to follow. Their Causal Shortcut Learning framework extracts step-by-step token chains from data and prioritizes them during masking. Across reasoning benchmarks and two base models, CSL beats SFT-variant baselines, with a reported 1.92% average gain over SFT-only models and up to 4.20% on MATH-500. ArXiv · AI/CL/LG's note
The authors argue that random masking leaves DLMs searching a much larger space than autoregressive models, making key reasoning tokens harder to follow. Their Causal Shortcut Learning framework extracts step-by-step token chains from data and prioritizes them during masking. Across reasoning benchmarks and two base models, CSL beats SFT-variant baselines, with a reported 1.92% average gain over SFT-only models and up to 4.20% on MATH-500. ArXiv · AI/CL/LG's note
score 5