DreamX: Video Diffusion Model is a Universal Latent Action Learner
DreamX compresses each latent video frame from 1,200 tokens to four while preserving action-relevant dynamics.
The method draws on a pretrained video diffusion model to separate actions from visual appearance. Its shared latent space supports zero-shot action transfer across human and robot embodiments and different environments, the authors report. DreamX achieves state-of-the-art performance among latent action models on LARY-Bench. The compact representation also reduces the computational cost of video-diffusion-based policies.
HF Daily Papers' note
The method draws on a pretrained video diffusion model to separate actions from visual appearance. Its shared latent space supports zero-shot action transfer across human and robot embodiments and different environments, the authors report. DreamX achieves state-of-the-art performance among latent action models on LARY-Bench. The compact representation also reduces the computational cost of video-diffusion-based policies.
HF Daily Papers' note
score 5