PIVOT: Efficient Query-Group Indexing for Token-Level Sparse Attention
PIVOT cuts the indexing bottleneck in sparse attention by sharing one prefix scan across nearby queries.
The paper says DSA-style sparse attention still pays an `O(L^2)` indexing cost because each query scores all prior tokens. PIVOT replaces that with a proxy query for a query group, then either reuses the proxy top-k tokens or refines them per query. The authors report matching dense DSA indexer accuracy on DeepSeek-V3.2 and GLM-5.1, with up to 4x faster indexing and up to 1.6x lower long-context end-to-end latency. Source: ArXiv · AI/CL/LG's note.
The paper says DSA-style sparse attention still pays an `O(L^2)` indexing cost because each query scores all prior tokens. PIVOT replaces that with a proxy query for a query group, then either reuses the proxy top-k tokens or refines them per query. The authors report matching dense DSA indexer accuracy on DeepSeek-V3.2 and GLM-5.1, with up to 4x faster indexing and up to 1.6x lower long-context end-to-end latency. Source: ArXiv · AI/CL/LG's note.
score 6