NeuronEye: Query-Guided Visual Concept Activation for Vision-Language Reasoning
NeuronEye turns sparse concept activations into an inference-time control layer for frozen vision-language models.
The paper says current VLMs bury object identity, layout, and attributes in entangled hidden states, making query-specific visual evidence hard to isolate. NeuronEye builds a sparse concept-level neuron vocabulary from intermediate representations, uses the text query to select relevant concept clusters and image patches, then feeds that focused evidence back into the vision tokens. It also suppresses dominant perceptual directions so weaker but relevant cues are not washed out. In reported tests on Qwen2.5-VL-7B, it improves CV-Bench overall accuracy by 3.1 points and BLINK Multi-view by 8.3 points, with similar trends on LLaVA-1.6-7B. ArXiv · AI/CL/LG's note
The paper says current VLMs bury object identity, layout, and attributes in entangled hidden states, making query-specific visual evidence hard to isolate. NeuronEye builds a sparse concept-level neuron vocabulary from intermediate representations, uses the text query to select relevant concept clusters and image patches, then feeds that focused evidence back into the vision tokens. It also suppresses dominant perceptual directions so weaker but relevant cues are not washed out. In reported tests on Qwen2.5-VL-7B, it improves CV-Bench overall accuracy by 3.1 points and BLINK Multi-view by 8.3 points, with similar trends on LLaVA-1.6-7B. ArXiv · AI/CL/LG's note
score 4