Aspire: Can Models Self-Evolve from Vague Goals?
ASPIRE tests whether agents can turn a vague capability goal into their own training plan, then survive hidden evaluation.
The benchmark gives only a natural-language goal while keeping downstream tasks hidden. Agents must choose data, update methods, validation signals, and when to evaluate. In the paper’s experiments, agents can run training and harness-editing loops, but weight-level improvements are rare and unstable. Local self-evaluations often do not transfer, and continued search can wipe out earlier gains. HF Daily Papers' note
The benchmark gives only a natural-language goal while keeping downstream tasks hidden. Agents must choose data, update methods, validation signals, and when to evaluate. In the paper’s experiments, agents can run training and harness-editing loops, but weight-level improvements are rare and unstable. Local self-evaluations often do not transfer, and continued search can wipe out earlier gains. HF Daily Papers' note
score 5