OccludeBench: Can VLMs Find What's Missing?
VLMs are weaker at using physical traces than scene context to identify fully hidden objects.
OccludeBench tests whether models can recover occluded objects from what remains visible in an image. Across five VLMs, accuracy averaged 42.7% when the answer depended on a direct cue such as a shadow or reflection, versus 62.7% when broader context was enough. Humans scored higher and showed no significant gap between the two case types. The authors trace the main failure to linking the visible projection back to the hidden object, and report that an inpaint-then-check intervention closed the gap for two image-generation-capable models. ArXiv · AI/CL/LG's note
OccludeBench tests whether models can recover occluded objects from what remains visible in an image. Across five VLMs, accuracy averaged 42.7% when the answer depended on a direct cue such as a shadow or reflection, versus 62.7% when broader context was enough. Humans scored higher and showed no significant gap between the two case types. The authors trace the main failure to linking the visible projection back to the hidden object, and report that an inpaint-then-check intervention closed the gap for two image-generation-capable models. ArXiv · AI/CL/LG's note
score 5