In-Distribution Forcing for Long Video Generation at Test Time
The paper’s fix is to stop polluted KV cache entries from being created during long rollouts.
The authors say long autoregressive video diffusion runs drift because cached key-value entries can move out of distribution past the training horizon. Their ID-Forcing framework keeps KV caching and conditioning aligned with training-time behavior. Its self-caching step stores each chunk without attending to prior KV entries, keeping the rolling window in distribution. They report minute-scale generation with less color, texture, and motion drift than prior KV-conditioning approaches. HF Daily Papers' note
The authors say long autoregressive video diffusion runs drift because cached key-value entries can move out of distribution past the training horizon. Their ID-Forcing framework keeps KV caching and conditioning aligned with training-time behavior. Its self-caching step stores each chunk without attending to prior KV entries, keeping the rolling window in distribution. They report minute-scale generation with less color, texture, and motion drift than prior KV-conditioning approaches. HF Daily Papers' note
score 5