From Truncation to Commitment: Persistent Context in Uniform Discrete Diffusion
The paper tests whether “chosen” diffusion tokens work better when they stay visible to later sampling steps.
Satoshi Hayakawa proposes committed reveal sampling, a training-free sampler for uniform discrete diffusion models that stores selected argmax tokens and feeds them back as context. The paper separates that persistent-context effect from top-p truncation and temperature changes in paired Duo-distilled experiments. CRS without top-p truncation reports lower generative perplexity than fixed `p=0.95` and `p=0.9` baselines across 8-64 function evaluations, with a better GenPPL-entropy tradeoff at 64 NFE. Other diversity and continuation metrics do not all rank the same operating points the same way. ArXiv · AI/CL/LG's note
Satoshi Hayakawa proposes committed reveal sampling, a training-free sampler for uniform discrete diffusion models that stores selected argmax tokens and feeds them back as context. The paper separates that persistent-context effect from top-p truncation and temperature changes in paired Duo-distilled experiments. CRS without top-p truncation reports lower generative perplexity than fixed `p=0.95` and `p=0.9` baselines across 8-64 function evaluations, with a better GenPPL-entropy tradeoff at 64 NFE. Other diversity and continuation metrics do not all rank the same operating points the same way. ArXiv · AI/CL/LG's note
score 4