Subliminal Clocks: Latent Time Modelling in Diffusion Language Models
Diffusion language models appear to carry an internal signal for how far denoising has progressed.
The paper says that signal is visible in the models’ residual streams and can be decoded with probes across layers. When researchers steer along the subspace tied to the inferred timestep, model confidence and entropy shift in predictable ways. They also describe the representation as structured and interpretable in activation space. HF Daily Papers' note
The paper says that signal is visible in the models’ residual streams and can be decoded with probes across layers. When researchers steer along the subspace tied to the inferred timestep, model confidence and entropy shift in predictable ways. They also describe the representation as structured and interpretable in activation space. HF Daily Papers' note
score 5