Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding
LatentStream turns streaming video history into a fixed-length latent working memory instead of just retrieving old frames as extra context.
The paper targets MLLMs that must answer questions over continuous video with causal access and bounded memory. Its system keeps short-, mid-, and long-term visual history, then uses latent tokens to pull evidence from those scopes and internalize it. A confidence-guided optimization step refines both the latent memory and retrieved evidence using predictive entropy. The authors report state-of-the-art results on online and offline video benchmarks. HF Daily Papers' note
The paper targets MLLMs that must answer questions over continuous video with causal access and bounded memory. Its system keeps short-, mid-, and long-term visual history, then uses latent tokens to pull evidence from those scopes and internalize it. A confidence-guided optimization step refines both the latent memory and retrieved evidence using predictive entropy. The authors report state-of-the-art results on online and offline video benchmarks. HF Daily Papers' note
score 4