RoMeRL: Balancing Feedback Coverage and the Memory-Reward Trap in Self-Evolving Agent Memory via Reduced-Order Utility States
RoMeRL compresses agent-memory utility tracking into fixed task-level states to keep feedback dense and limit reward contamination.
The paper targets self-evolving LLM agents whose memory systems become harder to train as interaction histories grow. Its method updates or replaces memories within fixed semantic coordinates, rather than letting trajectory-indexed utility states expand without bound. In tests on ALFWorld and LifelongAgentBench, the authors report better task performance, an 80.0% lower Cold-Q ratio, about 6.0x higher feedback density, 84.4% smaller maintained memory, and 21.1% fewer LLM calls. ArXiv · AI/CL/LG's note
The paper targets self-evolving LLM agents whose memory systems become harder to train as interaction histories grow. Its method updates or replaces memories within fixed semantic coordinates, rather than letting trajectory-indexed utility states expand without bound. In tests on ALFWorld and LifelongAgentBench, the authors report better task performance, an 80.0% lower Cold-Q ratio, about 6.0x higher feedback density, 84.4% smaller maintained memory, and 21.1% fewer LLM calls. ArXiv · AI/CL/LG's note
score 4