Latent Compaction with Test-Time Training
At 32× KV-cache compression, Latent Compaction reports 96.7% accuracy on RULER, versus 40.0% for KVZip+.
The method trains a compact cache for each document while keeping the model’s weights frozen. It first matches attention outputs and normalization values from the full cache, then refines the entire compact cache through end-to-end distillation. Training uses generic document prompts without access to downstream questions or answers. The authors report gains over compression baselines with the same cache budget on RULER, LongBench, and Phonebook Lookup.
HF Daily Papers' note
The method trains a compact cache for each document while keeping the model’s weights frozen. It first matches attention outputs and normalization values from the full cache, then refines the entire compact cache through end-to-end distillation. Training uses generic document prompts without access to downstream questions or answers. The authors report gains over compression baselines with the same cache budget on RULER, LongBench, and Phonebook Lookup.
HF Daily Papers' note
score 5