SKILL-KD: Contrastive Skill Distillation for LLM Agents
The paper turns a teacher agent’s successful run and a student agent’s failed run into explicit “skill patches” the weaker agent can reuse.
SKILL-KD compares both trajectories on the same task, distills the actionable gap into text guidance, then reruns the frozen student to see whether the patch works. If the student still fails, the patch is refined. The system also tracks edit histories and consolidates patches to avoid skill drift, choosing whether to add, change, delete, or skip rules. The authors report consistent gains across five agent benchmarks and two student settings versus fixed-model adaptation baselines. HF Daily Papers' note
SKILL-KD compares both trajectories on the same task, distills the actionable gap into text guidance, then reruns the frozen student to see whether the patch works. If the student still fails, the patch is refined. The system also tracks edit histories and consolidates patches to avoid skill drift, choosing whether to add, change, delete, or skip rules. The authors report consistent gains across five agent benchmarks and two student settings versus fixed-model adaptation baselines. HF Daily Papers' note
score 5