Automating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild?
Agents beat published best models across all 20 benchmarks, but their gains came almost entirely from ML engineering tricks.
Agent4Sci tests scientific ML work across 11 domains, with Claude Opus achieving a 10.4% mean relative improvement. The authors found recurring problems with judging novelty, using available resources and falling into local engineering sweeps. Adding skills for research ideation, experiment management and scheduled checklists raised Opus’s improvement to 20.1% and produced more diverse, domain-relevant interventions. No run was flagged for academic misconduct.
HF Daily Papers' note
Agent4Sci tests scientific ML work across 11 domains, with Claude Opus achieving a 10.4% mean relative improvement. The authors found recurring problems with judging novelty, using available resources and falling into local engineering sweeps. Adding skills for research ideation, experiment management and scheduled checklists raised Opus’s improvement to 20.1% and produced more diverse, domain-relevant interventions. No run was flagged for academic misconduct.
HF Daily Papers' note
score 6