ReRound: Reconstructive Rounding to Resolve Midpoint Ambiguity in Calibration-Free LLM Quantization
ReRound uses diffusion reconstructions only where low-bit rounding is ambiguous, then selects the tolerance by matching leading singular values.
The paper targets calibration-free post-training quantization for smaller LLMs at 3-bit and 4-bit weights. Reconstructed continuous weights guide rounding for values near quantization midpoints, while ordinary round-to-nearest handles weights closer to interval boundaries. The authors report that this beats standard RTN and other calibration-free methods, stays competitive with calibration-dependent approaches, and adds no inference-time overhead. ArXiv · AI/CL/LG's note
The paper targets calibration-free post-training quantization for smaller LLMs at 3-bit and 4-bit weights. Reconstructed continuous weights guide rounding for values near quantization midpoints, while ordinary round-to-nearest handles weights closer to interval boundaries. The authors report that this beats standard RTN and other calibration-free methods, stays competitive with calibration-dependent approaches, and adds no inference-time overhead. ArXiv · AI/CL/LG's note
score 4