Know It, Act on It: Investigating Memory Utilization in LLM Personalization
LLM agents can recall a user preference and still fail to act on it.
The paper separates memory recall from behavior with paired “Know” and “Act” tests on the same preference. Across 16 systems and five memory architectures, the authors evaluated 1,000 preferences at three levels of expression strength. They found a persistent gap: agents often knew the preference was present but did not tailor the response when it mattered. The weakness was especially pronounced for health and therapy-related preferences. ArXiv · AI/CL/LG's note
The paper separates memory recall from behavior with paired “Know” and “Act” tests on the same preference. Across 16 systems and five memory architectures, the authors evaluated 1,000 preferences at three levels of expression strength. They found a persistent gap: agents often knew the preference was present but did not tailor the response when it mattered. The weakness was especially pronounced for health and therapy-related preferences. ArXiv · AI/CL/LG's note
score 5