CorporateBench: Large-Scale Q&A Benchmarking with Temporal Knowledge Bases
The benchmark tests whether LLMs can reason over corporate-scale document collections without using real company data.
CorporateBench uses four synthetic firms, from 12 to 10,000 employees, with corpora exceeding 230,000 documents. The documents are generated from temporally evolving knowledge bases, so facts stay logically consistent across large collections. It evaluates information extraction and knowledge-base querying, and the authors report that five tested LLMs degrade as input size approaches realistic enterprise scale. ArXiv · AI/CL/LG's note
CorporateBench uses four synthetic firms, from 12 to 10,000 employees, with corpora exceeding 230,000 documents. The documents are generated from temporally evolving knowledge bases, so facts stay logically consistent across large collections. It evaluates information extraction and knowledge-base querying, and the authors report that five tested LLMs degrade as input size approaches realistic enterprise scale. ArXiv · AI/CL/LG's note
score 5