Bypassing inference bottlenecks: Accelerating complex AI search with Retrieve-for-Train
Google says the speedup comes from moving the expensive search “thinking” into offline training.
Retrieve-for-Train uses reinforcement learning to create fan-out search behavior, then distills it into a 53.9M-parameter diffusion retriever. The model generates a full set of target embeddings in one non-autoregressive pass, instead of producing sub-queries token by token at inference time. In Google’s tests on fashion and music retrieval tasks, it beat single-query search, zero-shot expansion, and a Best-of-N baseline on set-level measures. The post says the diffusion version delivered a 12 to 20x inference speedup over autoregressive approaches. Google Research's note
Retrieve-for-Train uses reinforcement learning to create fan-out search behavior, then distills it into a 53.9M-parameter diffusion retriever. The model generates a full set of target embeddings in one non-autoregressive pass, instead of producing sub-queries token by token at inference time. In Google’s tests on fashion and music retrieval tasks, it beat single-query search, zero-shot expansion, and a Best-of-N baseline on set-level measures. The post says the diffusion version delivered a 12 to 20x inference speedup over autoregressive approaches. Google Research's note
score 5