Token Time Continuous Diffusion for Language Modeling
TTCD gives diffusion language models separate clocks for separate tokens.
The paper proposes a continuous-space diffusion model that maps Gaussian noise deterministically into a final token canvas. Its key change is per-token timing, letting more certain tokens resolve faster while others keep refining. The authors say this avoids a source of error in fast discrete-space diffusion models, where many tokens are sampled in parallel. A 160M-parameter TTCD model trained on OpenWebText and self-distilled outperformed similar discrete models in conditional generation at high speedups, with similar gains reported on Sudoku solving. HF Daily Papers' note
The paper proposes a continuous-space diffusion model that maps Gaussian noise deterministically into a final token canvas. Its key change is per-token timing, letting more certain tokens resolve faster while others keep refining. The authors say this avoids a source of error in fast discrete-space diffusion models, where many tokens are sampled in parallel. A 160M-parameter TTCD model trained on OpenWebText and self-distilled outperformed similar discrete models in conditional generation at high speedups, with similar gains reported on Sudoku solving. HF Daily Papers' note
score 5