Hide and Seek: HIDE Concepts to SEEK explanations for black box models
HnS tests concept importance in black-box vision models by masking the concept and measuring how the output changes.
The method uses a vision-language model to find images aligned with a given concept, localizes that concept, hides it, and compares the model’s original and masked outputs. The authors say this works without model weights or concept-labeled datasets, and supports image-level attribution. They validated it on CelebA and COCO-Stuff, where it showed better agreement with ground-truth attributions than existing methods. They also demonstrate it on image captioning and satellite image segmentation, which they say current concept-attribution methods cannot handle. ArXiv · AI/CL/LG's note
The method uses a vision-language model to find images aligned with a given concept, localizes that concept, hides it, and compares the model’s original and masked outputs. The authors say this works without model weights or concept-labeled datasets, and supports image-level attribution. They validated it on CelebA and COCO-Stuff, where it showed better agreement with ground-truth attributions than existing methods. They also demonstrate it on image captioning and satellite image segmentation, which they say current concept-attribution methods cannot handle. ArXiv · AI/CL/LG's note
score 4