A Systematic Study of Semantic ID Spaces for Generative Information Retrieval
The paper tests what makes generated document IDs useful without retraining a full retrieval model each time.
Allal, Randrianarivo, and Lamprier frame semantic DocID design as the bottleneck in generative information retrieval. They unify Product Quantization, Residual Quantization, and hybrid variants into one design space, then vary properties such as hierarchy, parallelism, ID length, and codebook size. They also propose training-free intrinsic metrics for judging DocID quality and structural fidelity. The experiments cited are on MS MARCO 300K and NQ320K. ArXiv · AI/CL/LG's note
Allal, Randrianarivo, and Lamprier frame semantic DocID design as the bottleneck in generative information retrieval. They unify Product Quantization, Residual Quantization, and hybrid variants into one design space, then vary properties such as hierarchy, parallelism, ID length, and codebook size. They also propose training-free intrinsic metrics for judging DocID quality and structural fidelity. The experiments cited are on MS MARCO 300K and NQ320K. ArXiv · AI/CL/LG's note
score 4