Values Shape What Self-Improving Coding Agents Build
Agents given openness values scored better on held-out data; those given conservation values scored below agents given no values.
The study tested four value types through short statements in coding agents’ persistent instructions, keeping them fixed across multiple rounds of self-improvement. The resulting algorithms differed in size, complexity and approach. Letting agents revise their own values narrowed the differences between value sets, while values distilled from AI literature offered no detectable performance gain over no values. The authors conclude that human oversight remains necessary when designing those instructions.
ArXiv · AI/CL/LG's note
The study tested four value types through short statements in coding agents’ persistent instructions, keeping them fixed across multiple rounds of self-improvement. The resulting algorithms differed in size, complexity and approach. Letting agents revise their own values narrowed the differences between value sets, while values distilled from AI literature offered no detectable performance gain over no values. The authors conclude that human oversight remains necessary when designing those instructions.
ArXiv · AI/CL/LG's note
score 4