OmegaUse-OfficeVal: Benchmarking LLM Agents on Long-Horizon Office-Suite Tasks with Economic Grounding
The benchmark prices office-suite agent work against human labor, then finds current LLMs still short on quality.
OmegaUse-OfficeVal includes 100 long-horizon office-suite tasks adapted from practitioner requests through a privacy-preserving process. The paper says the tasks average 2.32 hours of human labor and are paired with labor-time and price-proxy signals for cost comparison. Frontier LLMs tested were cheaper and faster than humans, but did not approach human-level deliverable quality. HF Daily Papers' note
OmegaUse-OfficeVal includes 100 long-horizon office-suite tasks adapted from practitioner requests through a privacy-preserving process. The paper says the tasks average 2.32 hours of human labor and are paired with labor-time and price-proxy signals for cost comparison. Frontier LLMs tested were cheaper and faster than humans, but did not approach human-level deliverable quality. HF Daily Papers' note
score 5