When Memory Lies: An Empirical Study of Spatial Memory Staleness in VLM Agents
Stale spatial memory made a GPT-4o navigation agent die more than twice as often as having no memory.
The paper tests VLM agents in a changing FrozenLake environment where remembered spatial facts can conflict with current observations. Models that handled stale-memory checks in text did not necessarily ground the same conflicts visually, with vision F1 ranging from 0.887 to 0.067 on identical grids. A read-time audit filter reduced much of the safety cost in text mode, but unreliable visual auditing made filtering inconsistent. HF Daily Papers' note
The paper tests VLM agents in a changing FrozenLake environment where remembered spatial facts can conflict with current observations. Models that handled stale-memory checks in text did not necessarily ground the same conflicts visually, with vision F1 ranging from 0.887 to 0.067 on identical grids. A read-time audit filter reduced much of the safety cost in text mode, but unreliable visual auditing made filtering inconsistent. HF Daily Papers' note
score 4