SpecFirst: Behavioral Specification Elicitation as a First-Class Step in Agent-Based Program Synthesis from Scratch
SpecFirst separates behavioral probing from coding, and that split raised ProgramBench pass rates across all tested models.
The paper argues that from-scratch program synthesis fails when agents read docs, explore behavior, and write code in one loop. SpecFirst first has a spec agent query the execute-only binary and turn observations plus documentation into a structured behavioral specification. A separate coding agent then implements against that reference. On 200 ProgramBench tasks, the authors report pass-rate gains of 6.9% to 21.3% and exploration-coverage gains of 9.4% to 18.5%. Source: HF Daily Papers' note
The paper argues that from-scratch program synthesis fails when agents read docs, explore behavior, and write code in one loop. SpecFirst first has a spec agent query the execute-only binary and turn observations plus documentation into a structured behavioral specification. A separate coding agent then implements against that reference. On 200 ProgramBench tasks, the authors report pass-rate gains of 6.9% to 21.3% and exploration-coverage gains of 9.4% to 18.5%. Source: HF Daily Papers' note
score 4