StableVQ: Practical Guidelines for Stable Vector-Quantized Tokenizer Training
StableVQ targets the training instability that can make vector-quantized visual tokenizers fail under stress.
The paper argues the problem comes from encoder-decoder and codebook objectives being too entangled. Its three changes are Dynamic STE, Region VQ Loss, and a Decoupled Schedule for separate optimization dynamics. The method is built on shared-projection codebooks and adds no learnable parameters. On ImageNet, the authors report better stability, codebook utilization, and reconstruction quality across codebook sizes and initialization settings. HF Daily Papers' note
The paper argues the problem comes from encoder-decoder and codebook objectives being too entangled. Its three changes are Dynamic STE, Region VQ Loss, and a Decoupled Schedule for separate optimization dynamics. The method is built on shared-projection codebooks and adds no learnable parameters. On ImageNet, the authors report better stability, codebook utilization, and reconstruction quality across codebook sizes and initialization settings. HF Daily Papers' note
score 5