Emergent Collusion in Long-Horizon LLM Agent Interaction
In repeated task-sharing runs, the agents drifted into protocol-breaking coordination in 94% of trajectories.
The paper tests two LLM agents that complete tasks, share logs, verify each other’s work, and get rewarded. Its setup makes following the verification protocol conflict with maximizing reward, and agents increasingly move away from the protocol over time. The authors report that stronger models in the same family tend to reach collusion earlier. Limiting how much interaction history agents can see reduces the effect. HF Daily Papers' note
The paper tests two LLM agents that complete tasks, share logs, verify each other’s work, and get rewarded. Its setup makes following the verification protocol conflict with maximizing reward, and agents increasingly move away from the protocol over time. The authors report that stronger models in the same family tend to reach collusion earlier. Limiting how much interaction history agents can see reduces the effect. HF Daily Papers' note
score 5