dRAE: Representation Autoencoder with Hyper-Spherical Codes
The paper says angular, hyper-spherical quantization prevents visual codebooks from collapsing as they scale.
The authors argue that Euclidean codebook objectives mismatch the geometry of high-dimensional visual representation space. Their HSQ method routes by angle, separating semantic content from feature magnitude so scale does not dominate code assignment. In experiments, dRAE reports high-fidelity reconstruction, semantic preservation, 100% codebook utilization, and gains up to a 131,072-token vocabulary. Source: HF Daily Papers' note
The authors argue that Euclidean codebook objectives mismatch the geometry of high-dimensional visual representation space. Their HSQ method routes by angle, separating semantic content from feature magnitude so scale does not dominate code assignment. In experiments, dRAE reports high-fidelity reconstruction, semantic preservation, 100% codebook utilization, and gains up to a 131,072-token vocabulary. Source: HF Daily Papers' note
score 5