E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation
The benchmark tests whether LLM agents can run online stores for a simulated full year, not just complete short task chains.
E-Commerce Bench puts agents through 365 days of market research, supplier negotiation, inventory, sales, fulfillment, returns, and cash-flow decisions. The environment uses real e-commerce product and supplier data, with deterministic customers and supplier negotiations for reproducibility. In tests of 18 frontier models, GPT-5.6 Sol produced the highest year-end assets but ranked poorly on fraud avoidance and was not the most operationally efficient. Among open-weight models, Qwen3.8-Max-Preview led the group and showed the strongest improvement over time in bargaining down prices. HF Daily Papers' note
E-Commerce Bench puts agents through 365 days of market research, supplier negotiation, inventory, sales, fulfillment, returns, and cash-flow decisions. The environment uses real e-commerce product and supplier data, with deterministic customers and supplier negotiations for reproducibility. In tests of 18 frontier models, GPT-5.6 Sol produced the highest year-end assets but ranked poorly on fraud avoidance and was not the most operationally efficient. Among open-weight models, Qwen3.8-Max-Preview led the group and showed the strongest improvement over time in bargaining down prices. HF Daily Papers' note
score 5