DoSA: Dots Sparse Attention
DoSA tries to make sparse-attention indexing conditional without changing which tokens are selected.
The paper says Top-K selection does not always need full scores for every candidate; it only needs enough evidence to rule candidates out. DoSA uses bounds from a seed set, nonnegative head projections, residuals, and activation-informed aggregation to decide which candidates still need full scoring. The authors report up to 2.74x Indexer speedups over SGLang and up to 1.94x time-to-first-token speedups in end-to-end tests. HF Daily Papers' note
The paper says Top-K selection does not always need full scores for every candidate; it only needs enough evidence to rule candidates out. DoSA uses bounds from a seed set, nonnegative head projections, residuals, and activation-informed aggregation to decide which candidates still need full scoring. The authors report up to 2.74x Indexer speedups over SGLang and up to 1.94x time-to-first-token speedups in end-to-end tests. HF Daily Papers' note
score 5