A Vocabulary for Multi-Agent Automated Research Systems
The paper proposes a shared vocabulary for describing how multi-agent research systems are built, run, and judged.
It breaks those systems into choices about agents, operations, permissions, communication, visibility, action selection, initialization, and evaluation. A single run is treated as a trajectory, while repeated runs on the same task form a distribution because agents and setup can be stochastic. The author argues this makes design choices testable, including when agents communicate or retain information across runs. It also separates “taste” into generative taste, before scoring, and evaluative taste, the gap between proxy score and real quality. HF Daily Papers' note
It breaks those systems into choices about agents, operations, permissions, communication, visibility, action selection, initialization, and evaluation. A single run is treated as a trajectory, while repeated runs on the same task form a distribution because agents and setup can be stochastic. The author argues this makes design choices testable, including when agents communicate or retain information across runs. It also separates “taste” into generative taste, before scoring, and evaluative taste, the gap between proxy score and real quality. HF Daily Papers' note
score 4