A2Z GameSpec-Bench: How Faithfully Can Coding Agents Generate Games from Game Design Specifications?
The benchmark tests whether coding agents preserve long game-design requirements through working gameplay, not just code output.
A2Z GameSpec-Bench contains 100 long-form game design documents converted into dependency-aware contracts of rules, constraints, and prerequisite relations. The evaluation combines source-code inspection with scenario replay and adaptive playtesting, keeping the same contract across agents and revision rounds. The paper reports that current agents still struggle to satisfy interdependent requirements across implementation and actual play. Requirement-specific feedback improved GDD Fidelity by 10.9% over self-revision after two rounds. HF Daily Papers' note
A2Z GameSpec-Bench contains 100 long-form game design documents converted into dependency-aware contracts of rules, constraints, and prerequisite relations. The evaluation combines source-code inspection with scenario replay and adaptive playtesting, keeping the same contract across agents and revision rounds. The paper reports that current agents still struggle to satisfy interdependent requirements across implementation and actual play. Requirement-specific feedback improved GDD Fidelity by 10.9% over self-revision after two rounds. HF Daily Papers' note
score 5