UMM-JEPA: Generative Pretraining via Masked Next-Scale Representation Prediction for Unified Multimodal Models
UMM-JEPA scored 51.71 across 16 visual benchmarks, versus 46.36 for the understanding-only Qwen-VL baseline.
The model predicts continuous visual-understanding representations from coarse to fine, with pixel rendering added after representation pretraining. Controlled comparisons attribute most of the understanding gain to that prediction objective; a data-matched control left group means nearly unchanged. The authors report stronger, more consistent transfer to understanding than with discrete-code or latent-flow generation objectives. Additional tests indicate richer object and spatial structure and greater use of image-specific information.
HF Daily Papers' note
The model predicts continuous visual-understanding representations from coarse to fine, with pixel rendering added after representation pretraining. Controlled comparisons attribute most of the understanding gain to that prediction objective; a data-matched control left group means nearly unchanged. The authors report stronger, more consistent transfer to understanding than with discrete-code or latent-flow generation objectives. Additional tests indicate richer object and spatial structure and greater use of image-specific information.
HF Daily Papers' note
score 5