Sable: A Small-Field Sumcheck Argument for Bit-Exact Inference of Large Language Models
Sable proves a 4,096-token Qwen2.5-32B inference in 92.5 seconds on one GPU, at 41–50 times serving time.
The system verifies that a committed, integer-quantized transformer produced a given token sequence from a given prompt, bit for bit. That guarantee applies to its fixed integer computation: on WikiText-2, its greedy token matched the BF16 model at 86.3% of positions. A compiled verifier checks proofs on one CPU core in 19–56 seconds. The proofs are not zero-knowledge, so Sable targets open-weight models.
HF Daily Papers' note
The system verifies that a committed, integer-quantized transformer produced a given token sequence from a given prompt, bit for bit. That guarantee applies to its fixed integer computation: on WikiText-2, its greedy token matched the BF16 model at 86.3% of positions. A compiled verifier checks proofs on one CPU core in 19–56 seconds. The proofs are not zero-knowledge, so Sable targets open-weight models.
HF Daily Papers' note
score 5