CORE: Improving Compositional Reasoning in MLLM Embedding via Reranker Distillation
The paper says compositional retrieval improves when an embedding model is trained to imitate its own backbone acting as a reranker.
CORE builds candidate lists with five levels of compositional match, then uses a Rank-KL objective to transfer the reranker’s finer judgments into embeddings. The authors report Rank-KL beats contrastive learning and pairwise CoSENT under the same data and tuning budget. CORE-RERANKER-8B averages 82.7% across COLA, SUGARCREPE++, and NEGBENCH, while CORE-EMBED-8B posts the best embedding-model average in their comparison. The gains also transfer to MCMR without hurting COCO and Flickr30K retrieval. ArXiv · AI/CL/LG's note
CORE builds candidate lists with five levels of compositional match, then uses a Rank-KL objective to transfer the reranker’s finer judgments into embeddings. The authors report Rank-KL beats contrastive learning and pairwise CoSENT under the same data and tuning budget. CORE-RERANKER-8B averages 82.7% across COLA, SUGARCREPE++, and NEGBENCH, while CORE-EMBED-8B posts the best embedding-model average in their comparison. The gains also transfer to MCMR without hurting COCO and Flickr30K retrieval. ArXiv · AI/CL/LG's note
score 4