Context-weighted Discrete Flow Matching
The paper says local context can make discrete flow matching train and sample better.
Cherniavskii, Severo, and Ullrich modify the CTMC behind discrete flow matching to weight tokens by nearby context. They report negligible sampler overhead and a scaled cross-entropy loss that cuts generative perplexity by up to 63% on OpenWebText. The method also matches a strong semi-autoregressive block diffusion baseline while preserving arbitrary-order generation. ArXiv · AI/CL/LG's note
Cherniavskii, Severo, and Ullrich modify the CTMC behind discrete flow matching to weight tokens by nearby context. They report negligible sampler overhead and a scaled cross-entropy loss that cuts generative perplexity by up to 63% on OpenWebText. The method also matches a strong semi-autoregressive block diffusion baseline while preserving arbitrary-order generation. ArXiv · AI/CL/LG's note
score 5