SpecFirst: Behavioral Specification Elicitation as a First-Class Step in Agent-Based Program Synthesis from Scratch
SpecFirst makes agents write a behavioral spec before they start coding from scratch.
The paper argues that single-pass agent frameworks lose intent and carry early misunderstandings into implementation. SpecFirst splits the job: one agent probes an execute-only binary and turns observations plus documentation into a structured specification, then another agent codes from that reference. On 200 ProgramBench tasks, it improved test pass rates by 6.9% to 21.3% and binary exploration coverage by 9.4% to 18.5% across four models. ArXiv · AI/CL/LG's note
The paper argues that single-pass agent frameworks lose intent and carry early misunderstandings into implementation. SpecFirst splits the job: one agent probes an execute-only binary and turns observations plus documentation into a structured specification, then another agent codes from that reference. On 200 ProgramBench tasks, it improved test pass rates by 6.9% to 21.3% and binary exploration coverage by 9.4% to 18.5% across four models. ArXiv · AI/CL/LG's note
score 4