UniSpark: Exploring Hybrid Attention for Multimodal Understanding and Generation
UniSpark handles visual questions, image generation and editing in one model, with image synthesis reaching 2K resolution.
Its backbone combines full attention with GatedDeltaNet linear attention, using approximately 9 billion parameters in each branch. For interleaved text-image generation, it keeps noisy image targets from influencing later predictions while retaining clean visual context. Training follows a four-stage curriculum covering understanding and generation. The authors report scores of 2470.3 on MME-S and 0.83 on GenEval.
HF Daily Papers' note
Its backbone combines full attention with GatedDeltaNet linear attention, using approximately 9 billion parameters in each branch. For interleaved text-image generation, it keeps noisy image targets from influencing later predictions while retaining clean visual context. Training follows a four-stage curriculum covering understanding and generation. The authors report scores of 2470.3 on MME-S and 0.83 on GenEval.
HF Daily Papers' note
score 5