LongBanana: An Expert-Verified Benchmark for Long-Context Multi-Reference Image Synthesis
Each LongBanana task asks an image generator to reconcile about 32 reference images and 20 constraints in one output.
The benchmark contains 90 tasks with 4,670 expert-specified evaluation items. Its checklists assess content fidelity, correct entity relationships and satisfaction of overall constraints, including whether systems reject distracting references. Experts double-labeled 15% of the items, reaching 89.4% exact agreement. In the authors’ experiments, minimal agent workflows improved on direct generation; a planning, execution and reflection workflow delivered gains that depended on the harness.
ArXiv · AI/CL/LG's note
The benchmark contains 90 tasks with 4,670 expert-specified evaluation items. Its checklists assess content fidelity, correct entity relationships and satisfaction of overall constraints, including whether systems reject distracting references. Experts double-labeled 15% of the items, reaching 89.4% exact agreement. In the authors’ experiments, minimal agent workflows improved on direct generation; a planning, execution and reflection workflow delivered gains that depended on the harness.
ArXiv · AI/CL/LG's note
score 5