MotionGraphicsBench: Evaluating Motion Graphics Generation with Programmatic Verification
Programmatic checks agreed with human animation judgments 93.3% of the time, versus 83.3% for a GPT-6 Astra visual judge.
MotionGraphicsBench pairs 100 static SVGs and motion prompts with verification programs that report which requirements pass or fail while accepting different valid implementations. Across 47 configurations of 32 LLMs, first-attempt success ranged from 6.0% to 67.7%. Giving 15 models 32 attempts improved every model’s results, but success still ranged from 38.0% to 95.0%. Common failures included matching reference poses, following requested trajectories and coordinating related events.
HF Daily Papers' note
MotionGraphicsBench pairs 100 static SVGs and motion prompts with verification programs that report which requirements pass or fail while accepting different valid implementations. Across 47 configurations of 32 LLMs, first-attempt success ranged from 6.0% to 67.7%. Giving 15 models 32 attempts improved every model’s results, but success still ranged from 38.0% to 95.0%. Common failures included matching reference poses, following requested trajectories and coordinating related events.
HF Daily Papers' note
score 4