Get Hint-Augmented GRPO Done Right Through Activation Steering
ASH learns a steering vector from training hints, then applies it at inference without putting hints in the prompt.
Hint-augmented GRPO addresses a training dead end: when every generated answer fails, zero rewards provide no learning signal. But removing those hints at inference creates a mismatch that the authors say makes gains inconsistent. ASH uses hinted and plain questions to learn a shared behavioral shift, then injects the resulting vector into the model’s activations. Across four models, the authors report improved pass@k on competition mathematics and out-of-distribution benchmarks, with average pass@8 gains of 2.60% and 2.21% for the two Qwen3 models.
ArXiv · AI/CL/LG's note
Hint-augmented GRPO addresses a training dead end: when every generated answer fails, zero rewards provide no learning signal. But removing those hints at inference creates a mismatch that the authors say makes gains inconsistent. ASH uses hinted and plain questions to learn a shared behavioral shift, then injects the resulting vector into the model’s activations. Across four models, the authors report improved pass@k on competition mathematics and out-of-distribution benchmarks, with average pass@8 gains of 2.60% and 2.21% for the two Qwen3 models.
ArXiv · AI/CL/LG's note
score 4