UEmbed: Unified Sparse and Dense Multimodal Embeddings
UEmbed produces sparse lexical and dense multimodal embeddings from a single decoder-only model pass.
The paper says it adds learnable special tokens, splits the vocabulary into subsets, and uses causal hidden states to build a full sparse vector alongside dense representations. The authors release 2B, 4B, and 9B versions trained on public data. UEmbed-9B scores 71.8 dense and 71.0 sparse on MMEB-v2, and is reported as competitive on BEIR. HF Daily Papers' note
The paper says it adds learnable special tokens, splits the vocabulary into subsets, and uses causal hidden states to build a full sparse vector alongside dense representations. The authors release 2B, 4B, and 9B versions trained on public data. UEmbed-9B scores 71.8 dense and 71.0 sparse on MMEB-v2, and is reported as competitive on BEIR. HF Daily Papers' note
score 5