MoI-Bench: A Benchmark for Image Generation and Editing on Mobile Devices
The authors find that mobile image models need broader task coverage more than further gains in deployment efficiency.
MoI-Bench tests 64 subtasks across 1,150 samples, pairing operations such as removal and replacement with visual targets such as portraits and text. Its MoI-Eval framework assesses instruction following, visual quality and detail preservation using task-specific questions. A two-dimensional quality–deployability map compares model capabilities with practical mobile deployment costs. The evaluation highlights text replacement, identity preservation and text rendering as areas needing improvement.
ArXiv · AI/CL/LG's note
MoI-Bench tests 64 subtasks across 1,150 samples, pairing operations such as removal and replacement with visual targets such as portraits and text. Its MoI-Eval framework assesses instruction following, visual quality and detail preservation using task-specific questions. A two-dimensional quality–deployability map compares model capabilities with practical mobile deployment costs. The evaluation highlights text replacement, identity preservation and text rendering as areas needing improvement.
ArXiv · AI/CL/LG's note
score 4