Successive Capacity Growth: Task-Complexity-Driven Width and Depth Expansion for Vision Transformer Encoders in JEPA World Models
SCG lets a JEPA vision encoder start tiny and expand only when a task shows it needs more capacity.
The paper proposes adding attention heads or transformer blocks through a test-and-verify loop that can roll back changes if prediction loss does not improve. In the reported tasks, the adaptive encoder expanded depth for multi-object dynamics and width for 2D navigation. It improved prediction loss over fixed baselines while using far fewer parameters than a large fixed model. The author also reports zero false-positive expansions and bit-exact function preservation during trials. ArXiv · AI/CL/LG's note
The paper proposes adding attention heads or transformer blocks through a test-and-verify loop that can roll back changes if prediction loss does not improve. In the reported tasks, the adaptive encoder expanded depth for multi-object dynamics and width for 2D navigation. It improved prediction loss over fixed baselines while using far fewer parameters than a large fixed model. The author also reports zero false-positive expansions and bit-exact function preservation during trials. ArXiv · AI/CL/LG's note
score 4