Stratified Sparse Attention: Reducing the Sparse-to-Dense Gap at Extreme Sparsity
At 50× attention sparsity, SSA retained 85% of dense-attention performance on a 128K-context test.
SSA keeps high-scoring tokens and samples the remaining KV cache using estimated attention scores, correcting for each token’s inclusion probability. On RULER-HARD with Ministral-3-8B, it raised accuracy from 63.8% to 69.3% at 128K context against the strongest practical baseline. The authors report improvements over baselines at matched sparsity across four benchmarks and five models spanning 3B to 27B parameters. Optimized CUDA kernels delivered up to 2.96× decode speedup over FlashInfer at 1M context.
HF Daily Papers' note
SSA keeps high-scoring tokens and samples the remaining KV cache using estimated attention scores, correcting for each token’s inclusion probability. On RULER-HARD with Ministral-3-8B, it raised accuracy from 63.8% to 69.3% at 128K context against the strongest practical baseline. The authors report improvements over baselines at matched sparsity across four benchmarks and five models spanning 3B to 27B parameters. Optimized CUDA kernels delivered up to 2.96× decode speedup over FlashInfer at 1M context.
HF Daily Papers' note
score 5