ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement
The paper’s claim is that harness self-improvement transfers better when changes are modular, contrastive, and trained away from the evaluation benchmarks.
ModularRSI compares successful and failed trajectories on the same task, then aggregates patterns across tasks to find recurring harness weaknesses. It limits changes to five separate harness modules before integrating them back into one system. The authors say they built 2,000 executable evolution tasks disjoint from the downstream benchmarks. In tests on TB2.0 and SWE-Bench Verified, the evolved harness improved on unseen tasks and transferred across foundation models. HF Daily Papers' note
ModularRSI compares successful and failed trajectories on the same task, then aggregates patterns across tasks to find recurring harness weaknesses. It limits changes to five separate harness modules before integrating them back into one system. The authors say they built 2,000 executable evolution tasks disjoint from the downstream benchmarks. In tests on TB2.0 and SWE-Bench Verified, the evolved harness improved on unseen tasks and transferred across foundation models. HF Daily Papers' note
score 5