Unfolding the Leech Lattice: Fused Multi-Shell Decoding and VRAM Layouts for 2-Bit LLM Weights
The paper implements the missing multi-shell decoder needed to serve Leech-lattice 2-bit LLM weights, then measures the VRAM cost.
It reports a fused dequantize-plus-matvec path for the full 301-class codebook, verified against f64. In batch-1 decode GEMV, binary bit-plane layouts beat one-hot masks, landing at 4.80 bits per weight and 2.15x FP16. The author says a trellis kernel still reads 2.40x fewer bytes and runs 2.27x faster than the served layout, with the gap tracking memory traffic. The preprint also reports end-to-end gains across 4B, 8B, and 14B models, alongside quality costs that shrink with size. ArXiv · AI/CL/LG's note
It reports a fused dequantize-plus-matvec path for the full 301-class codebook, verified against f64. In batch-1 decode GEMV, binary bit-plane layouts beat one-hot masks, landing at 4.80 bits per weight and 2.15x FP16. The author says a trellis kernel still reads 2.40x fewer bytes and runs 2.27x faster than the served layout, with the gap tracking memory traffic. The preprint also reports end-to-end gains across 4B, 8B, and 14B models, alongside quality costs that shrink with size. ArXiv · AI/CL/LG's note
score 4