Parameter Exploration for RLVR via Variational Learning
The paper tests policy sampling in parameter space as an exploration lever for LLM reinforcement learning.
The authors introduce Perturbed Parameter Policy Optimization, or 3PO, as an alternative to action-space controls such as temperature scaling. In experiments on OLMo-3-1025-7B and Qwen2.5-Math-7B, the methods improved average downstream results over standard GRPO at nearly the same FLOPs cost. The paper also reports fewer zero-advantage groups and fewer malformed or incorrect rollouts when using multiple parameter samples. Source: ArXiv · AI/CL/LG's note.
The authors introduce Perturbed Parameter Policy Optimization, or 3PO, as an alternative to action-space controls such as temperature scaling. In experiments on OLMo-3-1025-7B and Qwen2.5-Math-7B, the methods improved average downstream results over standard GRPO at nearly the same FLOPs cost. The paper also reports fewer zero-advantage groups and fewer malformed or incorrect rollouts when using multiple parameter samples. Source: ArXiv · AI/CL/LG's note.
score 5