SWAM: Scaling World Action Models with Cross-Modal Alignment
SWAM reports 40% success across three real-world tasks, versus 23.33% for the strongest baseline.
The researchers studied roughly 47,000 hours of video and demonstrations to examine how learning actions can weaken a model’s language-video alignment. Under a fixed training budget, mixing egocentric video with UMI demonstrations improved control generalization. After post-training only on RoboTwin Clean demonstrations, the model achieved 88.10% success in Clean-to-Clean tests and 61.56% in Clean-to-Random tests. The authors say they release the curated data, training code and model weights.
HF Daily Papers' note
The researchers studied roughly 47,000 hours of video and demonstrations to examine how learning actions can weaken a model’s language-video alignment. Under a fixed training budget, mixing egocentric video with UMI demonstrations improved control generalization. After post-training only on RoboTwin Clean demonstrations, the model achieved 88.10% success in Clean-to-Clean tests and 61.56% in Clean-to-Random tests. The authors say they release the curated data, training code and model weights.
HF Daily Papers' note
score 5