Any-OPD: Heterogeneous On-Policy Distillation for Flow-Matching Models via Representation-Space Bridging
Any-OPD lets a smaller flow-matching image model learn from a mismatched black-box teacher without sharing latents, architecture, or timesteps.
The paper says standard on-policy distillation breaks when teacher and student models use different latent spaces or schedules. Its method compares independently decoded outputs inside a frozen vision representation, then aligns training by continuous noise level rather than step index. In the reported test, distilling 12B FLUX.1-dev into 2.5B SD3.5-Medium raised PickScore from 0.846 to 0.884 and HPSv3 from 9.12 to 10.97. Direct latent regression, the authors say, failed to train at all. HF Daily Papers' note
The paper says standard on-policy distillation breaks when teacher and student models use different latent spaces or schedules. Its method compares independently decoded outputs inside a frozen vision representation, then aligns training by continuous noise level rather than step index. In the reported test, distilling 12B FLUX.1-dev into 2.5B SD3.5-Medium raised PickScore from 0.846 to 0.884 and HPSv3 from 9.12 to 10.97. Direct latent regression, the authors say, failed to train at all. HF Daily Papers' note
score 5