Exploring Weight-Space Programming: Capabilities, Generalization, and Limitations
Editing just 1,023 parameters lifted MATH-500 accuracy by 29.45 percentage points in a held-out evaluation.
Weight-Space Programming lets reasoning agents inspect and modify model weights directly, offering an alternative to gradient-based adaptation. Researchers tested 16 editing trajectories across seven benchmarks and two Qwen3.5 model sizes. One math-edited checkpoint also improved GPQA Diamond, IFEval and SciBench without further editing, but left SciCode unchanged. Results varied substantially: six trajectories retained the original weights because candidate improvements could not be reliably validated.
HF Daily Papers' note
Weight-Space Programming lets reasoning agents inspect and modify model weights directly, offering an alternative to gradient-based adaptation. Researchers tested 16 editing trajectories across seven benchmarks and two Qwen3.5 model sizes. One math-edited checkpoint also improved GPQA Diamond, IFEval and SciBench without further editing, but left SciCode unchanged. Results varied substantially: six trajectories retained the original weights because candidate improvements could not be reliably validated.
HF Daily Papers' note
score 5