Distill Skills into Weights, Not Prompts: Abstract Skills as Privileged Signals for On-Policy Self-Distillation
SKALD turns temporary skill-card guidance into model weights, so the student needs no privileged prompt at test time.
The paper says standard verifiable-reward RL loses its training signal when rollout groups are all right or all wrong, which made up 63.0-68.0% of groups in their experiments. SKALD pairs a question-only student with the same Qwen3-Base model as a teacher, conditioned on an abstract skill card filtered to avoid explicit answers. Distillation is gated to cases where verified rollouts show a positive teacher advantage, and uses an annealed tilted objective to manage mismatch between teacher-favored tokens and student likelihood. Across five held-out math benchmarks, it beats GRPO in avg@8 by +2.46, +4.85, and +12.01 at 0.6B, 1.7B, and 4B. ArXiv · AI/CL/LG's note
The paper says standard verifiable-reward RL loses its training signal when rollout groups are all right or all wrong, which made up 63.0-68.0% of groups in their experiments. SKALD pairs a question-only student with the same Qwen3-Base model as a teacher, conditioned on an abstract skill card filtered to avoid explicit answers. Distillation is gated to cases where verified rollouts show a positive teacher advantage, and uses an annealed tilted objective to manage mismatch between teacher-favored tokens and student likelihood. Across five held-out math benchmarks, it beats GRPO in avg@8 by +2.46, +4.85, and +12.01 at 0.6B, 1.7B, and 4B. ArXiv · AI/CL/LG's note
score 5