PAWBench: How Far Are We from Probabilistically Aligned World Modeling?
The benchmark says current video generators miss the probability distribution of possible outcomes, not just individual realism.
PAWBench tests whether repeated video generations recover the range and likelihood of valid physical behaviors from the same setup. The authors pair it with PAWEval, which turns rollouts into empirical outcome distributions across 50 scenarios. In tests of eleven current systems, no model consistently matched the reference probabilities while also covering the valid behaviors. They also probe whether prompts, noise sampling, or training can shift those distributions. ArXiv · AI/CL/LG's note
PAWBench tests whether repeated video generations recover the range and likelihood of valid physical behaviors from the same setup. The authors pair it with PAWEval, which turns rollouts into empirical outcome distributions across 50 scenarios. In tests of eleven current systems, no model consistently matched the reference probabilities while also covering the valid behaviors. They also probe whether prompts, noise sampling, or training can shift those distributions. ArXiv · AI/CL/LG's note
score 5