Omni-InteractScope: Benchmarking Proactive and Agentic Interaction in Continuous Omnimodal Streams
None of the 12 tested models delivered an on-time first response on more than 40% of tool-invocation targets.
Omni-InteractScope tests continuous audiovisual assistance through 1,479 human-annotated interactions spanning eight task groups. The best combined timing-and-tool accuracy was 32.0%, while timely recall for reminders without explicit questions never exceeded 23.0%. Nine models completed none of the 26 guidance sequences; the other three covered just one to three sequences each. Control experiments also showed that tool-invocation scores alone do not establish whether actions were grounded in observed events.
HF Daily Papers' note
Omni-InteractScope tests continuous audiovisual assistance through 1,479 human-annotated interactions spanning eight task groups. The best combined timing-and-tool accuracy was 32.0%, while timely recall for reminders without explicit questions never exceeded 23.0%. Nine models completed none of the 26 guidance sequences; the other three covered just one to three sequences each. Control experiments also showed that tool-invocation scores alone do not establish whether actions were grounded in observed events.
HF Daily Papers' note
score 4