PriceBench: A Diagnostic Benchmark for Price, Quality, and Brand Preferences in LLM Booking Agents
The benchmark finds that LLM booking agents can steer the same hotel task toward very different prices.
PriceBench tests 28 models from eight providers on 3,600 booking tasks using real New York City hotel properties. The paper says stronger models are more consistent in their choices, but not necessarily aligned on what they prefer. Across models, price sensitivity varies by more than an order of magnitude, shifting mean booked nightly price from $247 to $393 on identical tasks. The author releases the tasks, code, and all model responses. ArXiv · AI/CL/LG's note
PriceBench tests 28 models from eight providers on 3,600 booking tasks using real New York City hotel properties. The paper says stronger models are more consistent in their choices, but not necessarily aligned on what they prefer. Across models, price sensitivity varies by more than an order of magnitude, shifting mean booked nightly price from $247 to $393 on identical tasks. The author releases the tasks, code, and all model responses. ArXiv · AI/CL/LG's note
score 5