SparseEngine: Sparse-First Inference Engine
The paper claims a sparse-first serving design can cut long-context inference costs without forcing every sparse method into one cache layout.
SparseEngine gives sparse attention methods their own KV representation and computation path while sharing serving-state transitions. It supports 15 methods across four categories, according to the abstract. The authors also describe Chain Cache for resuming KV-eviction methods from retained history, plus Prefix-Cache Pruning for removing selected KV regions while preserving logical-prefix matching. They report over 10x higher throughput with KV eviction, over 2.5x faster decoding than vLLM at matched concurrency, and over 2x end-to-end speedup on agent benchmarks. HF Daily Papers' note
SparseEngine gives sparse attention methods their own KV representation and computation path while sharing serving-state transitions. It supports 15 methods across four categories, according to the abstract. The authors also describe Chain Cache for resuming KV-eviction methods from retained history, plus Prefix-Cache Pruning for removing selected KV regions while preserving logical-prefix matching. They report over 10x higher throughput with KV eviction, over 2.5x faster decoding than vLLM at matched concurrency, and over 2x end-to-end speedup on agent benchmarks. HF Daily Papers' note
score 5