Beyond Dyadic Memory: Interaction-Aware Multimodal Memory with Adaptive Agentic Retrieval for Multi-Party Spoken Conversations
VoxPolyMem is built to remember not just what was said, but who said it, who they were addressing, and how that carries across sessions.
The paper proposes an interaction-aware multimodal memory system for multi-party spoken conversations, combining speaker identification with layered memories for interactions, facts, and participant profiles. Its retrieval agent rewrites queries and chooses tools or memory layers as it gathers evidence. The authors also introduce VoxPolyBench to test memory evolution, personalized answers, retrieval reasoning, and attribution. VoxPolyMem scores 85.0 overall on that benchmark, 23.6 points above the strongest evaluated baseline. HF Daily Papers' note
The paper proposes an interaction-aware multimodal memory system for multi-party spoken conversations, combining speaker identification with layered memories for interactions, facts, and participant profiles. Its retrieval agent rewrites queries and chooses tools or memory layers as it gathers evidence. The authors also introduce VoxPolyBench to test memory evolution, personalized answers, retrieval reasoning, and attribution. VoxPolyMem scores 85.0 overall on that benchmark, 23.6 points above the strongest evaluated baseline. HF Daily Papers' note
score 4