ReproBridge: Building Reproducibility into ML Research Artifacts
Refined research artifacts lifted agents’ reproduction success from 76.4% to 92.6%, while cutting downstream API spending by 19.7% on average.
ReproBridge preserves recovered procedures in a refined codebase, environment setup, experiment launchers and a reproduction guide. The benchmark covers 1,029 tasks from 387 experiments across 36 papers, with a separate auditor checking execution evidence. All four evaluated models exceeded 90% mean success with refined artifacts, narrowing the gap between models from 21.6 to 3.9 percentage points. Removing any artifact component reduced success; removing code or environment setup caused the largest losses.
HF Daily Papers' note
ReproBridge preserves recovered procedures in a refined codebase, environment setup, experiment launchers and a reproduction guide. The benchmark covers 1,029 tasks from 387 experiments across 36 papers, with a separate auditor checking execution evidence. All four evaluated models exceeded 90% mean success with refined artifacts, narrowing the gap between models from 21.6 to 3.9 percentage points. Removing any artifact component reduced success; removing code or environment setup caused the largest losses.
HF Daily Papers' note
score 5