Spectral-Driven Adaptive-Rank Optimizer for Large Language Model Reinforcement Learning with Verifiable Rewards
The paper argues RLVR gradients collapse into a smaller useful subspace, and builds an optimizer around that changing rank.
Chen, Zhao, and Wen report that this spectral pattern appears in RLVR training, not in supervised fine-tuning, and mostly disappears when verifiable rewards are replaced with random ones. Their optimizer, Ares, estimates the effective rank from gradient energy and restricts updates to that low-rank subspace as training evolves. Across model scales, tasks, and RLVR algorithms, they say Ares improves accuracy over commonly used optimizers. HF Daily Papers' note
Chen, Zhao, and Wen report that this spectral pattern appears in RLVR training, not in supervised fine-tuning, and mostly disappears when verifiable rewards are replaced with random ones. Their optimizer, Ares, estimates the effective rank from gradient energy and restricts updates to that low-rank subspace as training evolves. Across model scales, tasks, and RLVR algorithms, they say Ares improves accuracy over commonly used optimizers. HF Daily Papers' note
score 5