Coupled Calibration and Learning: Mitigating Teacher Bias in LLM Distillation without Target-Domain Reward Feedback
The paper proposes a distillation method meant to correct a biased teacher using only source-domain reward feedback.
Coupled Calibration and Learning calibrates the teacher on source questions, then uses that calibrated teacher to train the student on target questions. The student’s updates feed back into later calibration steps. The authors prove convergence to an oracle student under their autoregressive policy framework, measured by expected average KL divergence. They also show regularized direct matching can remain wrong even when the teacher has higher regularized target reward than any student policy. ArXiv · AI/CL/LG's note
Coupled Calibration and Learning calibrates the teacher on source questions, then uses that calibrated teacher to train the student on target questions. The student’s updates feed back into later calibration steps. The authors prove convergence to an oracle student under their autoregressive policy framework, measured by expected average KL divergence. They also show regularized direct matching can remain wrong even when the teacher has higher regularized target reward than any student policy. ArXiv · AI/CL/LG's note
score 4