StreamEval: Evaluating Interactive Streaming Video Generation Models
Strong visuals did not guarantee that video models followed requests or preserved state over time.
StreamEval tested 37 models on 300 scripted cases lasting 15–300 seconds across avatar dialogue, avatar performance, world generation and two editing tasks. Each case introduces multimodal requests over time, specifying what must change and what must remain intact. On a 0–100 scale, the highest task-specific scores included 36.90 for following avatar-performance requests, 25.93 for following text-conditioned world-generation requests and 15.00 for reference-editing success. The authors identify reliable execution under timing and preservation constraints as a central challenge.
HF Daily Papers' note
StreamEval tested 37 models on 300 scripted cases lasting 15–300 seconds across avatar dialogue, avatar performance, world generation and two editing tasks. Each case introduces multimodal requests over time, specifying what must change and what must remain intact. On a 0–100 scale, the highest task-specific scores included 36.90 for following avatar-performance requests, 25.93 for following text-conditioned world-generation requests and 15.00 for reference-editing success. The authors identify reliable execution under timing and preservation constraints as a central challenge.
HF Daily Papers' note
score 5