PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers
PrismQuant targets the rotation problem inside grouped INT4 quantization, not just the size of activation outliers.
The paper says quantization quality depends on how activation energy lines up with the quantizer’s group structure. Its method aligns the leading activation eigenspace with the constant group subspace and gives a closed-form optimal solution for that objective. Tests cover Llama, Qwen, and Mistral models up to 70B, plus a 30B MoE model. The authors report near-full-precision zero-shot accuracy on Llama-3.1-70B and deployment speedups over matched FP16 baselines. HF Daily Papers' note
The paper says quantization quality depends on how activation energy lines up with the quantizer’s group structure. Its method aligns the leading activation eigenspace with the constant group subspace and gives a closed-form optimal solution for that objective. Tests cover Llama, Qwen, and Mistral models up to 70B, plus a 30B MoE model. The authors report near-full-precision zero-shot accuracy on Llama-3.1-70B and deployment speedups over matched FP16 baselines. HF Daily Papers' note
score 5