Branching On-Policy Distillation
BranchOPD widens on-policy distillation by branching from uncertain student decisions instead of spending samples on fully separate rollouts.
The method chooses branch points using student uncertainty and teacher-student disagreement, then supervises both the original trajectory and the new continuations. The paper says shared-prefix reuse cuts student-generated token costs by about 19% on ALFWorld and 31% on WebShop versus seven independent trajectories. It also adds protocol-boundary repair to reduce repetition and wasted generation under task constraints. Reported gains beat Vanilla OPD and Multi-Rollout OPD across the evaluated math and agent benchmarks. HF Daily Papers' note
The method chooses branch points using student uncertainty and teacher-student disagreement, then supervises both the original trajectory and the new continuations. The paper says shared-prefix reuse cuts student-generated token costs by about 19% on ALFWorld and 31% on WebShop versus seven independent trajectories. It also adds protocol-boundary repair to reduce repetition and wasted generation under task constraints. Reported gains beat Vanilla OPD and Multi-Rollout OPD across the evaluated math and agent benchmarks. HF Daily Papers' note
score 5