STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization
STEPQuant targets the recurrent-state memory cost in linear attention without taking the usual accuracy hit from low-bit quantization.
The paper says quantization errors matter differently depending on how long a state persists and which key rows they affect. STEPQuant uses that spatial-temporal structure to assign precision and fit row and column scales for Delta-rule recurrent states. In tests on Qwen3.8-27B and Kimi-Linear-48B-A3B-Instruct, it nearly matches FP32-state accuracy at a nominal 6-bit budget. With SGLang GPU kernels, the authors report more than 5x recurrent-state compression and up to 68.7% lower total serving memory. HF Daily Papers' note
The paper says quantization errors matter differently depending on how long a state persists and which key rows they affect. STEPQuant uses that spatial-temporal structure to assign precision and fit row and column scales for Delta-rule recurrent states. In tests on Qwen3.8-27B and Kimi-Linear-48B-A3B-Instruct, it nearly matches FP32-state accuracy at a nominal 6-bit budget. With SGLang GPU kernels, the authors report more than 5x recurrent-state compression and up to 68.7% lower total serving memory. HF Daily Papers' note
score 4