Canonical Color as a Lens into Concept Decodability in Vision Encoders and VLMs
The paper tests whether vision encoders retain an object’s expected color even when the image is grayscale.
The authors build a dataset of objects with canonical colors and probe encoders on both color and grayscale inputs. They find canonical color is still linearly decodable from grayscale images, and that signal tracks with predicted object identity. In full VLMs, post-training can substantially change how decodable color is inside the vision encoder. ArXiv · AI/CL/LG's note
The authors build a dataset of objects with canonical colors and probe encoders on both color and grayscale inputs. They find canonical color is still linearly decodable from grayscale images, and that signal tracks with predicted object identity. In full VLMs, post-training can substantially change how decodable color is inside the vision encoder. ArXiv · AI/CL/LG's note
score 4