ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding
The paper says streaming video models can avoid full-depth processing on every incoming frame by using shallow-layer KV cache as a live retrieval index.
ShallowStream keeps a lightweight always-on index while frames arrive, then uses shallow-layer attention at query time to pick evidence frames. Its selection step is diversity-aware, meant to retrieve both precise and broad context for answering. The authors report performance comparable to the strongest existing streaming methods, with per-frame prefill latency reduced by up to 52.1x and 10-second end-to-end latency by up to 11.9x. Code is listed as available. HF Daily Papers' note
ShallowStream keeps a lightweight always-on index while frames arrive, then uses shallow-layer attention at query time to pick evidence frames. Its selection step is diversity-aware, meant to retrieve both precise and broad context for answering. The authors report performance comparable to the strongest existing streaming methods, with per-frame prefill latency reduced by up to 52.1x and 10-second end-to-end latency by up to 11.9x. Code is listed as available. HF Daily Papers' note
score 5