Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL
The paper claims masked diffusion models can simulate RL text environments more coherently than much larger autoregressive LMs.
Darshan Deshpande frames text world modeling as steerable transition dynamics over state, task context, tools, rules, and directives. The work uses 239,403 grounded state-action trajectories across nine open-source environments and twelve frontier model families. In comparisons, masked diffusion language models reportedly improve coherence, groundedness, and rollout diversity at similar latency. A GRPO setup with deterministic state checks shows up to 47% absolute gains on out-of-distribution environments without environment-specific fine-tuning. HF Daily Papers' note
Darshan Deshpande frames text world modeling as steerable transition dynamics over state, task context, tools, rules, and directives. The work uses 239,403 grounded state-action trajectories across nine open-source environments and twelve frontier model families. In comparisons, masked diffusion language models reportedly improve coherence, groundedness, and rollout diversity at similar latency. A GRPO setup with deterministic state checks shows up to 47% absolute gains on out-of-distribution environments without environment-specific fine-tuning. HF Daily Papers' note
score 5