ConvergeFlow: Language Flow with Provable Convergence to Token Embeddings
The paper claims a flow-based language model can land directly on valid token embeddings without a cross-entropy decoder.
ConvergeFlow constrains its data predictor to the convex hull of token embeddings and trains with the MSE objective from flow matching. Under stated regularity conditions, the authors prove convergence to valid token embeddings even with predictor errors. They also describe three sampling mechanisms to tune the perplexity–entropy trade-off. OpenWebText experiments are reported as competitive with existing continuous and discrete diffusion language models. ArXiv · AI/CL/LG's note
ConvergeFlow constrains its data predictor to the convex hull of token embeddings and trains with the MSE objective from flow matching. Under stated regularity conditions, the authors prove convergence to valid token embeddings even with predictor errors. They also describe three sampling mechanisms to tune the perplexity–entropy trade-off. OpenWebText experiments are reported as competitive with existing continuous and discrete diffusion language models. ArXiv · AI/CL/LG's note
score 5