A Frozen Pixel-Space Diffusion Model Can Guide Itself with Its Own Samples
A frozen pixel diffusion model can improve its own sampling with a lightweight head trained on synthetic outputs.
The paper’s method, Synthetic Self-Guidance, reads coarse structure from an intermediate transformer layer and compares it with the model’s final prediction during sampling. The backbone stays frozen; only the prediction head is trained. The authors say generated samples were enough to train that head, and performed better than real images for this purpose. Across ImageNet tests, they report lower FID with under 1% of full-model training compute. HF Daily Papers' note
The paper’s method, Synthetic Self-Guidance, reads coarse structure from an intermediate transformer layer and compares it with the model’s final prediction during sampling. The backbone stays frozen; only the prediction head is trained. The authors say generated samples were enough to train that head, and performed better than real images for this purpose. Across ImageNet tests, they report lower FID with under 1% of full-model training compute. HF Daily Papers' note
score 5