MixTok: Adaptive Visual Tokenization with Global 1D Tokens and Sparse 2D Refinement
MixTok spends 2D tokens only where an image needs local detail.
The paper proposes a hybrid tokenizer that pairs compact global 1D tokens with sparse 2D refinement tokens. A content-adaptive router ranks image locations and chooses how many 2D tokens to use under an average token budget. Those selected 2D features replace recovered spatial features before a shared decoder reconstructs the image. On ImageNet 256 x 256, the authors report rFID 1.195 and gFID 1.95 at about 128 and 131 content tokens per image. HF Daily Papers' note
The paper proposes a hybrid tokenizer that pairs compact global 1D tokens with sparse 2D refinement tokens. A content-adaptive router ranks image locations and chooses how many 2D tokens to use under an average token budget. Those selected 2D features replace recovered spatial features before a shared decoder reconstructs the image. On ImageNet 256 x 256, the authors report rFID 1.195 and gFID 1.95 at about 128 and 131 content tokens per image. HF Daily Papers' note
score 4