Keep-or-Drop? Adaptive Tokenizer for Compact Video Representation
The paper’s core claim is that video tokenizers can compress more aggressively by learning which latent tokens are worth keeping.
KATok is a transformer-based VAE with an adaptive token selector trained alongside the latent tokens. It scores tokens by content-richness and drops uninformative ones, allowing compression to vary with the video’s spatio-temporal complexity. The authors add cascaded and joint position-prediction strategies to reduce spatial misalignment caused by token dropping. They report strong reconstruction and generation quality at a state-of-the-art compression ratio, attributing the gains to reduced redundancy in video data. HF Daily Papers' note
KATok is a transformer-based VAE with an adaptive token selector trained alongside the latent tokens. It scores tokens by content-richness and drops uninformative ones, allowing compression to vary with the video’s spatio-temporal complexity. The authors add cascaded and joint position-prediction strategies to reduce spatial misalignment caused by token dropping. They report strong reconstruction and generation quality at a state-of-the-art compression ratio, attributing the gains to reduced redundancy in video data. HF Daily Papers' note
score 4