OSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific Software
The paper tests whether VLM agents can actually operate scientific tools and produce checkable research artifacts.
OSWorld-Science includes 146 expert-shaped tasks across areas such as molecular drawing, retrosynthesis, pathology image analysis, statistical computing, and physical simulation. Its evaluators inspect outputs like molecules, masks, plots, application states, and numerical results, with partial credit for unfinished work. The authors report that 12 current VLMs, even with a strong agent harness, still struggle with key scientific-software workflows. HF Daily Papers' note
OSWorld-Science includes 146 expert-shaped tasks across areas such as molecular drawing, retrosynthesis, pathology image analysis, statistical computing, and physical simulation. Its evaluators inspect outputs like molecules, masks, plots, application states, and numerical results, with partial credit for unfinished work. The authors report that 12 current VLMs, even with a strong agent harness, still struggle with key scientific-software workflows. HF Daily Papers' note
score 5