Read, Allocate, Route: Role-Coupled Tokenization for Pixel-Space Generative Transformers
SpecSlot adds fine image detail without lengthening the sequence used for global attention.
The model splits each spatial anchor into frequency slots, sending only the lowest-frequency slot through global attention while keeping interactions among all slots local. On ImageNet-512, the authors report a 2.84-point FID improvement over a compute-matched JiT control with 3.9% fewer FLOPs. In text-to-image tests, it beats JiT-B/8 by 1.06 points on DPG-Bench at 40.6% lower forward cost. The paper is under review at ICLR 2027.
ArXiv · AI/CL/LG's note
The model splits each spatial anchor into frequency slots, sending only the lowest-frequency slot through global attention while keeping interactions among all slots local. On ImageNet-512, the authors report a 2.84-point FID improvement over a compute-matched JiT control with 3.9% fewer FLOPs. In text-to-image tests, it beats JiT-B/8 by 1.06 points on DPG-Bench at 40.6% lower forward cost. The paper is under review at ICLR 2027.
ArXiv · AI/CL/LG's note
score 5