Learning to Read the Contextual Tokens in Diffusion Transformers
The paper claims diffusion transformers’ updated text tokens can be read as a live description of the image forming inside the model.
The authors train a small bottleneck network to translate intermediate contextual tokens into a frozen LLM’s input space, then ask the LLM questions about the emerging image. They report that these tokens expose global scene semantics early in denoising, with finer detail becoming readable later. The signal remains decodable even with an empty prompt, suggesting the tokens absorb image-specific information from the visual stream itself. They also introduce Contextual Alignment, a training method meant to strengthen that visual-semantic signal and improve generation quality and coverage. HF Daily Papers' note
The authors train a small bottleneck network to translate intermediate contextual tokens into a frozen LLM’s input space, then ask the LLM questions about the emerging image. They report that these tokens expose global scene semantics early in denoising, with finer detail becoming readable later. The signal remains decodable even with an empty prompt, suggesting the tokens absorb image-specific information from the visual stream itself. They also introduce Contextual Alignment, a training method meant to strengthen that visual-semantic signal and improve generation quality and coverage. HF Daily Papers' note
score 5