Self-OPD: On-Policy Distillation for Flow Matching Models without Teacher
Self-OPD trains flow matching models with the student’s own sampled branches instead of a separate teacher.
At each timestep, the method creates stochastic SDE candidates, rolls them out, and scores them against a deterministic self-reference baseline. High-advantage branches pull the model; low-advantage branches push it away, with normalization to control variance. For mixed objectives, it combines normalized reward scores rather than mixing gradients directly. The paper reports stronger results than prior RL and OPD methods on single- and multi-reward benchmarks. HF Daily Papers' note
At each timestep, the method creates stochastic SDE candidates, rolls them out, and scores them against a deterministic self-reference baseline. High-advantage branches pull the model; low-advantage branches push it away, with normalization to control variance. For mixed objectives, it combines normalized reward scores rather than mixing gradients directly. The paper reports stronger results than prior RL and OPD methods on single- and multi-reward benchmarks. HF Daily Papers' note
score 5