SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation
SANA-Video 2.0 claims 720p video generation on a single H100 by mixing mostly linear attention with periodic softmax anchors.
The paper presents 5B and 14B video diffusion transformers built around a 3:1 linear-to-softmax attention pattern. Its attention residuals carry completed block summaries forward, which the authors say improves deep-layer effective rank by about 12%. With 40-step sampling, the model reports an 84.30 VBench score in 13.2 seconds at 480p on one H100. The authors also report a 3.2x faster compiled DiT forward pass than a matched full-softmax baseline at 720p/60s, plus further gains from Sol-Engine optimization.
HF Daily Papers' note
The paper presents 5B and 14B video diffusion transformers built around a 3:1 linear-to-softmax attention pattern. Its attention residuals carry completed block summaries forward, which the authors say improves deep-layer effective rank by about 12%. With 40-step sampling, the model reports an 84.30 VBench score in 13.2 seconds at 480p on one H100. The authors also report a 3.2x faster compiled DiT forward pass than a matched full-softmax baseline at 720p/60s, plus further gains from Sol-Engine optimization.
HF Daily Papers' note
score 5