SteerDistill: Quantizing LLM Agents via Closed-Loop Teacher Steering
SteerDistill’s quantized agents stayed within one point of full-precision performance across five benchmarks and three models.
The method uses a full-precision teacher to help a quantized student revise failed responses, refining its guidance with feedback from the student’s execution. Those corrections are distilled into the student so it can run without guidance. Under W4A16 quantization, the preprint reports average scores 1.67–2.18 points above the strongest quantized baseline. On τ²-Bench, it beat QeRL’s success rate by 1.84 percentage points while using 67% fewer training GPU hours.
HF Daily Papers' note
The method uses a full-precision teacher to help a quantized student revise failed responses, refining its guidance with feedback from the student’s execution. Those corrections are distilled into the student so it can run without guidance. Under W4A16 quantization, the preprint reports average scores 1.67–2.18 points above the strongest quantized baseline. On τ²-Bench, it beat QeRL’s success rate by 1.84 percentage points while using 67% fewer training GPU hours.
HF Daily Papers' note
score 5