DominoBench: Evaluating Multi-Step Physical Reasoning from Still Images
Frontier models got both versions of a minimally changed physical scene right in at most 34% of pairs without help.
DominoBench tested 30 models across 10 families on 854 still images of simulated chain reactions. Its 427 pairs shift one quantity by one to three grid steps, changing whether a person survives. Pointing to the decisive step or supplying exact geometry helped little. Providing the deciding calculation and its inputs—including speeds invisible in a still image—raised the strongest models’ accuracy on the affected person to 82–92%, while small open-weight models stayed near chance.
HF Daily Papers' note
DominoBench tested 30 models across 10 families on 854 still images of simulated chain reactions. Its 427 pairs shift one quantity by one to three grid steps, changing whether a person survives. Pointing to the decisive step or supplying exact geometry helped little. Providing the deciding calculation and its inputs—including speeds invisible in a still image—raised the strongest models’ accuracy on the affected person to 82–92%, while small open-weight models stayed near chance.
HF Daily Papers' note
score 5