Megadose The AI curve, compressed.

The Noise Floor of a Quantization Comparison: Engine, restart, batch and kernel drift, measured against 8-bit and 4-bit effects in two decision models

· HF Daily Papers ·
Nearly unchanged accuracy masked hundreds of worse decisions in one 4-bit build.

Across 31,275 public questions, Clef’s Q4_K_M build lost a net six correct answers, while 436 individual decisions got worse—an exploratory finding. In a controlled 2,400-decision comparison of Cloudflare’s Clef and Clef-flash models, engine, restart, batching and kernel changes shifted up to 0.92% of decisions. Four-bit quantization shifted 3.50%–5.79%, with every measured interval excluding every non-quantization source. FP8 changes also exceeded restart noise, contradicting one of the authors’ pre-registered predictions.

HF Daily Papers' note

score 5

Categories: Research