Megadose The AI curve, compressed.

WeightBench: Evaluating Language Models as Weight-Space Programmers

· ArXiv · AI/CL/LG ·
Frontier agents can write working control policies from weights alone, but still fail at patching language models and spotting impossible edits.

WeightBench tests direct manipulation of neural-network parameters without gradient descent across 42 tasks in five tracks. The target networks range from 2,000-parameter control policies to 7-billion-parameter language models. Agents could read small networks and write working control policies without running them, but could not compress language models without data or reliably distinguish impossible edits from difficult ones. GPT-6.1-sol scored 0.568 on the 38 tasks it attempted, against Claude Opus 5.5’s 0.546 on that same subset.

ArXiv · AI/CL/LG's note

score 5

Categories: Research