Where Should LLM Agents Adapt at Test Time? In Context, in Weights, or Both
Retaining experience in context delivered the biggest standalone gain; adding reinforcement learning pushed mean success to 51.6%.
The study tested Qwen3.5-9B on 109 tasks across four environments, evaluating each task before the agent could learn from it. The best in-context policy raised mean success from 28.8% to 49.4%, while adding GRPO reached 51.6%; self-distillation produced mixed results. Context retention reduced lookups and shortened attempts, but agents also frequently skipped task-specific inspection and verification. Adding GRPO helped restore those checks while preserving fewer environment lookups.
ArXiv · AI/CL/LG's note
The study tested Qwen3.5-9B on 109 tasks across four environments, evaluating each task before the agent could learn from it. The best in-context policy raised mean success from 28.8% to 49.4%, while adding GRPO reached 51.6%; self-distillation produced mixed results. Context retention reduced lookups and shortened attempts, but agents also frequently skipped task-specific inspection and verification. Adding GRPO helped restore those checks while preserving fewer environment lookups.
ArXiv · AI/CL/LG's note
score 5