HyQuant: Hybrid-Precision Quantization for LLM Attention
HyQuant keeps most attention data low-bit, but protects a small set of high-impact tokens and local states in full precision.
The paper targets the error spike that low-bit quantization can cause inside LLM attention. HyQuant selects vertical-line tokens and local-window states using lightweight attention-pattern signals, then leaves those regions high precision while quantizing the rest. It applies the design in prefill and decode, including KV-cache compression with fused dequantization and attention computation. The authors report nearly lossless accuracy across tasks, models, and datasets. HF Daily Papers' note
The paper targets the error spike that low-bit quantization can cause inside LLM attention. HyQuant selects vertical-line tokens and local-window states using lightweight attention-pattern signals, then leaves those regions high precision while quantizing the rest. It applies the design in prefill and decode, including KV-cache compression with fused dequantization and attention computation. The authors report nearly lossless accuracy across tasks, models, and datasets. HF Daily Papers' note
score 4