The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metric
The paper proposes TPIPS, a perceptual metric that can judge image similarity according to a text-specified aspect.
The authors argue that standard similarity scores flatten judgments like shape, color, or other semantic cues into one number. They build a dataset of human judgments over image triplets, annotated with free-form aspects of similarity. Frontier vision-language models still fall short of human consensus on the benchmark. A fine-tuned VLM using that data better matches human perception and is shown on retrieval, compositional search, and generative-model evaluation. ArXiv · AI/CL/LG's note
The authors argue that standard similarity scores flatten judgments like shape, color, or other semantic cues into one number. They build a dataset of human judgments over image triplets, annotated with free-form aspects of similarity. Frontier vision-language models still fall short of human consensus on the benchmark. A fine-tuned VLM using that data better matches human perception and is shown on retrieval, compositional search, and generative-model evaluation. ArXiv · AI/CL/LG's note
score 5