MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations
The authors report quantizing a 122-billion-parameter model in 64 minutes using 32.5GB of GPU memory.
MACaQ quantizes weights and activations to four bits, processing each linear layer independently so only one layer needs to fit in memory at a time. Its Flip-Refinement procedure adjusts individual quantized values using Hessian information to improve accuracy. On Llama-3-8B, it matches DuQuant++ and MR-GPTQ’s best perplexity in 11 times less quantization time. Activation transforms add 2.3% end-to-end inference overhead over pure MXFP4 on Qwen3-32B in vLLM.
HF Daily Papers' note
MACaQ quantizes weights and activations to four bits, processing each linear layer independently so only one layer needs to fit in memory at a time. Its Flip-Refinement procedure adjusts individual quantized values using Hessian information to improve accuracy. On Llama-3-8B, it matches DuQuant++ and MR-GPTQ’s best perplexity in 11 times less quantization time. Activation transforms add 2.3% end-to-end inference overhead over pure MXFP4 on Qwen3-32B in vLLM.
HF Daily Papers' note
score 5