SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions
The paper proposes one Bayesian framework that prunes and quantizes neural network weights at the same time.
SQS uses a spike-and-slab prior for sparsity and Gaussian mixture models to represent low-bit quantized weights. The authors say an approximation makes the otherwise intractable objective workable with minimal accuracy loss. Experiments cover ResNet, BERT-base, Llama3.2, and Qwen2.5, with higher compression rates than listed baselines while keeping comparable performance drops. The paper is accepted at TMLR. HF Daily Papers' note
SQS uses a spike-and-slab prior for sparsity and Gaussian mixture models to represent low-bit quantized weights. The authors say an approximation makes the otherwise intractable objective workable with minimal accuracy loss. Experiments cover ResNet, BERT-base, Llama3.2, and Qwen2.5, with higher compression rates than listed baselines while keeping comparable performance drops. The paper is accepted at TMLR. HF Daily Papers' note
score 4