QuantMLA: Function-Aligned Dual-Path Quantization for Low-Bit MLA KV Caching
QuantMLA reports joint INT4 caching for both MLA content and RoPE paths with little accuracy loss.
The paper models how quantization errors behave differently across MLA’s content and RoPE cache paths, with RoPE-path errors described as especially amplified. Its method learns path-specific transformations that can be fused into model parameters offline, avoiding online transformation overhead. Across four MLA model families, the authors say QuantMLA keeps competitive accuracy, including an INT2 content cache paired with INT4 RoPE keys on reasoning and code benchmarks. They also report 3.59x physical cache compression at 128K context and 5.168x higher whole-job output throughput than BF16 in a cache-pressure serving workload. HF Daily Papers' note
The paper models how quantization errors behave differently across MLA’s content and RoPE cache paths, with RoPE-path errors described as especially amplified. Its method learns path-specific transformations that can be fused into model parameters offline, avoiding online transformation overhead. Across four MLA model families, the authors say QuantMLA keeps competitive accuracy, including an INT2 content cache paired with INT4 RoPE keys on reasoning and code benchmarks. They also report 3.59x physical cache compression at 128K context and 5.168x higher whole-job output throughput than BF16 in a cache-pressure serving workload. HF Daily Papers' note
score 4