Megadose The AI curve, compressed.

Accelerate Bench: Evaluating Agents for Application-Level Program Optimization

· HF Daily Papers ·
Most agents produced correct programs; the gap was how much faster they made them.

Accelerate Bench tests agents on 32 CPU optimization tasks drawn from scientific applications and supercomputing competitions. Among 16 model-and-harness pairings, the top two achieved at least 2× speedups on 22 or more tasks; the other fourteen did so on 8–17. Controlled replays showed identical source code running at 0.29× to 2.12× baseline speed depending only on build and runtime settings. One code change delivered a 1.01× or 6.4× speedup depending on another change.

HF Daily Papers' note

score 5

Categories: Research