SemMask: Semantic-Prioritized Progressive Masking for Diffusion Language Models
SemMask trains diffusion language models to establish meaning earlier, with reported gains in accuracy using fewer decoding steps.
The authors identify a function-word-first bias that can limit global planning in text writing and code generation. Their method uses model gradients to identify high-information tokens, then prioritizes those tokens during masking and adjusts their loss weights to encourage earlier prediction. They report quality comparable to strong autoregressive baselines, with better semantic coherence. The supplied abstract gives no numerical results or decoding-step counts.
ArXiv · AI/CL/LG's note
The authors identify a function-word-first bias that can limit global planning in text writing and code generation. Their method uses model gradients to identify high-information tokens, then prioritizes those tokens during masking and adjusts their loss weights to encourage earlier prediction. They report quality comparable to strong autoregressive baselines, with better semantic coherence. The supplied abstract gives no numerical results or decoding-step counts.
ArXiv · AI/CL/LG's note
score 4