MAP: A Benchmark on Multimodal Accessibility Planning for Real World Places
MAP tests whether multimodal AI can verify real-world accessibility claims before someone visits a place.
The benchmark gives systems requests to confirm or recommend points of interest with specific accessibility requirements. It includes claim verification and visual evidence retrieval, checking both stated place information and supporting images. The authors say the ground truth can be refreshed on a schedule because place and accessibility data change over time. Evaluation uses automatic ratings plus human ratings for some responses. ArXiv · AI/CL/LG's note
The benchmark gives systems requests to confirm or recommend points of interest with specific accessibility requirements. It includes claim verification and visual evidence retrieval, checking both stated place information and supporting images. The authors say the ground truth can be refreshed on a schedule because place and accessibility data change over time. Evaluation uses automatic ratings plus human ratings for some responses. ArXiv · AI/CL/LG's note
score 4