VC-Attention: Value Smoothing and Softmax Casting for Low-bit Attention
VC-Attention targets the two blockers for low-bit diffusion attention: value outliers and the softmax bottleneck.
The paper proposes a training-free framework that clusters value tokens within hardware blocks, quantizes residuals after subtracting the block mean, and restores the mean through the online softmax row sum. It also replaces the FP32 exponential and format conversion with ExpCast-FP8, mapping log-domain scores directly into E4M3 probability codes. The authors report kernel speedups over BF16 FlashAttention-4 of 1.46–1.59x on datacenter Blackwell and Hopper GPUs, and 2.3–3.6x on workstation cards. End-to-end video generation is reported as 1.13–1.19x faster on datacenter GPUs and 1.36–1.70x faster on workstation cards. HF Daily Papers' note
The paper proposes a training-free framework that clusters value tokens within hardware blocks, quantizes residuals after subtracting the block mean, and restores the mean through the online softmax row sum. It also replaces the FP32 exponential and format conversion with ExpCast-FP8, mapping log-domain scores directly into E4M3 probability codes. The authors report kernel speedups over BF16 FlashAttention-4 of 1.46–1.59x on datacenter Blackwell and Hopper GPUs, and 2.3–3.6x on workstation cards. End-to-end video generation is reported as 1.13–1.19x faster on datacenter GPUs and 1.36–1.70x faster on workstation cards. HF Daily Papers' note
score 5