RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction
RestoreKV adds a small learned “restore cache” to recover accuracy after severe KV cache eviction.
The method keeps the existing scorer and eviction rule, then uses a few restore tokens after prefill to attend to the full cache once and generate a compact context-conditioned supplement. Its LoRA adapters are turned off for later queries and decoding, and training uses self-distillation from the frozen full-cache model while updating only 0.4% of parameters. The paper reports gains across four backbones and four long-context benchmarks, including KVzip rising from 38.2 to 73.2 on RULER-4K at a 5% budget for Qwen3-4B. It also reports less than 0.5% one-time cache-construction overhead in a 32K-context evaluation. HF Daily Papers' note
The method keeps the existing scorer and eviction rule, then uses a few restore tokens after prefill to attend to the full cache once and generate a compact context-conditioned supplement. Its LoRA adapters are turned off for later queries and decoding, and training uses self-distillation from the frozen full-cache model while updating only 0.4% of parameters. The paper reports gains across four backbones and four long-context benchmarks, including KVzip rising from 38.2 to 73.2 on RULER-4K at a 5% budget for Qwen3-4B. It also reports less than 0.5% one-time cache-construction overhead in a 32K-context evaluation. HF Daily Papers' note
score 5