VideoMemBench: How Well Do MLLMs Remember in Video Conversations?
Claude Opus 5 led more than 20 tested models on VideoMemBench, scoring just 45.7% overall.
The benchmark tests whether video models can reuse what they saw and said earlier in a conversation. It contains 326 videos and 2,476 dialogue turns, including 1,387 that depend on earlier exchanges. Those dependencies require prior context without repeating the information needed to answer; each dialogue was reviewed by human experts. The authors have released the dataset and evaluation.
HF Daily Papers' note
The benchmark tests whether video models can reuse what they saw and said earlier in a conversation. It contains 326 videos and 2,476 dialogue turns, including 1,387 that depend on earlier exchanges. Those dependencies require prior context without repeating the information needed to answer; each dialogue was reviewed by human experts. The authors have released the dataset and evaluation.
HF Daily Papers' note
score 4