Error Certificates for KV-Cache Eviction via Randomized Design
Randomized KV-cache eviction can attach an error certificate; deterministic eviction cannot.
The paper argues that top-k eviction leaves the serving system unable to identify how much attention-output error the dropped tokens caused. Its randomized design keeps a Poisson-sampled tail and uses a Hájek correction plus a variance estimator to produce a per-step certificate. In tests, that certificate did not beat output log-probability for predicting failure and did not make budget escalation useful. Its useful role was attribution: separating cache-induced failures from inherent ones and choosing recomputation better than random or confidence gating. ArXiv · AI/CL/LG's note
The paper argues that top-k eviction leaves the serving system unable to identify how much attention-output error the dropped tokens caused. Its randomized design keeps a Poisson-sampled tail and uses a Hájek correction plus a variance estimator to produce a per-step certificate. In tests, that certificate did not beat output log-probability for predicting failure and did not make budget escalation useful. Its useful role was attribution: separating cache-induced failures from inherent ones and choosing recomputation better than random or confidence gating. ArXiv · AI/CL/LG's note
score 5