Stage-Replay Divergence Follows the KV Cache: Fixed-Prefix Precision Controls and Bidirectional Cache Transplantation
The paper says replaying the same tokens can diverge because the live KV cache carries state that prefill reconstruction does not.
In a 200-item Qwen2.5-derived test, BF16 live-cache continuations differed from one-shot prefill on 166 suffixes and 20 correctness labels, even though exact replicas stayed exact. When the same token states were held fixed, BF16 disagreements returned, while FP32 produced no decoded disagreement. Transplanting all 48 key/value layers made divergent continuations follow the cache donor in every tested case. ArXiv · AI/CL/LG's note
In a 200-item Qwen2.5-derived test, BF16 live-cache continuations differed from one-shot prefill on 166 suffixes and 20 correctness labels, even though exact replicas stayed exact. When the same token states were held fixed, BF16 disagreements returned, while FP32 produced no decoded disagreement. Transplanting all 48 key/value layers made divergent continuations follow the cache donor in every tested case. ArXiv · AI/CL/LG's note
score 5