LOCKS: Page-Local Compact Key Summaries for Efficient Long-Context Decoding
LOCKS claims FullKV-level quality at 100K+ context while attending about 2% of tokens.
The paper targets the KV-cache bottleneck in long-context decoding by giving each page a compact local key summary. It uses those summaries to estimate attention mass and select only the top pages, without reading candidate keys or values during selection. The abstract says it stays close to full-cache quality on LongBench-v1, tracks an oracle on RULER, and posts its biggest gains on AIME26 and MATH-500. It is described as a drop-in vLLM plugin with CUDA-graph batched decode. ArXiv · AI/CL/LG's note
The paper targets the KV-cache bottleneck in long-context decoding by giving each page a compact local key summary. It uses those summaries to estimate attention mass and select only the top pages, without reading candidate keys or values during selection. The abstract says it stays close to full-cache quality on LongBench-v1, tracks an oracle on RULER, and posts its biggest gains on AIME26 and MATH-500. It is described as a drop-in vLLM plugin with CUDA-graph batched decode. ArXiv · AI/CL/LG's note
score 5