Registers Matter for Pixel-Space Diffusion Transformers
Register tokens help pixel-space DiTs even without the ViT-style token outliers they were built to fix.
The paper says Diffusion Transformers do not show the high-norm patch-token outliers seen in ViTs, but still gain from registers. The effect is stronger in pixel-space DiTs than in latent-space DiTs. The authors trace the benefit to cleaner intermediate feature maps at high noise levels, which may improve visual structure. They also propose Register Guidance to amplify register-token contributions tied to coherence. HF Daily Papers' note
The paper says Diffusion Transformers do not show the high-norm patch-token outliers seen in ViTs, but still gain from registers. The effect is stronger in pixel-space DiTs than in latent-space DiTs. The authors trace the benefit to cleaner intermediate feature maps at high noise levels, which may improve visual structure. They also propose Register Guidance to amplify register-token contributions tied to coherence. HF Daily Papers' note
score 5