WebArena-Pro: A Heterogeneous, Multimodal, Reproducible Benchmark for Web Agents
The best agents scored 62.3% on 300 browser tasks that include audio and video.
WebArena-Pro spans 20 self-hosted applications across six domain categories. It treats audio and video as core observations alongside text and images, with dedicated actions for extracting information from each. Each task runs in isolation to support reproducible, parallel evaluation without live-site drift. Tasks pass through LLM-assisted triage and human validation before release.
HF Daily Papers' note
WebArena-Pro spans 20 self-hosted applications across six domain categories. It treats audio and video as core observations alongside text and images, with dedicated actions for extracting information from each. Each task runs in isolation to support reproducible, parallel evaluation without live-site drift. Tasks pass through LLM-assisted triage and human validation before release.
HF Daily Papers' note
score 5