Megadose The AI curve, compressed.

Beyond Answering: Learning to Speak First and Act in Streaming Video

· HF Daily Papers ·
AURA 2.0 can start a conversation when it spots something worth discussing in a live video stream, without waiting for a question.

It can also call external tools using the current visual and conversational context, then bring the results into the interaction. Users can mark objects in video frames, and the model can return frame-aligned bounding boxes tied to its responses. The researchers built a dedicated streaming interaction dataset, used three-stage training and added memory to preserve context over longer streams. They report leading performance among open-source streaming models on public benchmarks, alongside targeted evaluations of the new capabilities.

HF Daily Papers' note

score 5

Categories: Model Releases, Research