World Models' Last Exam in Physics
The benchmark tests video generators against measurable physics, and the top model still scored only 57.76/100.
The paper introduces 40 controlled tasks across mechanics, optics, fluids, thermal and phase-change effects, electromagnetism, and surface tension. Each task uses an initial image, a prompt, and predefined physical criteria instead of reference videos. The evaluator screens whether the task is observable, then applies task-specific quantitative measurements. Across eight video generation models and 1,280 videos, the authors report persistent physical inconsistencies and wide variation by task. HF Daily Papers' note
The paper introduces 40 controlled tasks across mechanics, optics, fluids, thermal and phase-change effects, electromagnetism, and surface tension. Each task uses an initial image, a prompt, and predefined physical criteria instead of reference videos. The evaluator screens whether the task is observable, then applies task-specific quantitative measurements. Across eight video generation models and 1,280 videos, the authors report persistent physical inconsistencies and wide variation by task. HF Daily Papers' note
score 5