HABIT-Bench: Evaluating Habit Induction from Longitudinal Weak Evidence in Agent Memory
The best memory method scored 33.61% macro accuracy when asked to infer and apply users’ habits.
HABIT-Bench tests eight memory methods using synthetic histories spanning hundreds of sessions across food, finance, software and travel. Its four-choice tasks include temporary exceptions, changing habits and suggestions users never accepted. The researchers evaluated the methods with Qwen3-235B-A22B and GLM-5.3-Flash. Performance remained limited even when annotated evidence was supplied directly, indicating that access to past interactions alone does not ensure appropriate use of user habits.
HF Daily Papers' note
HABIT-Bench tests eight memory methods using synthetic histories spanning hundreds of sessions across food, finance, software and travel. Its four-choice tasks include temporary exceptions, changing habits and suggestions users never accepted. The researchers evaluated the methods with Qwen3-235B-A22B and GLM-5.3-Flash. Performance remained limited even when annotated evidence was supplied directly, indicating that access to past interactions alone does not ensure appropriate use of user habits.
HF Daily Papers' note
score 4