Learning Skills from Historical Action Trajectories: Action Experience Dictionary for World Action Models
The paper proposes an Action Experience Dictionary that lets world action models reuse stored action trajectories across manipulation tasks.
The authors say current WAMs predict visual dynamics and actions but lack an explicit way to carry prior action experience into new tasks. AED encodes historical physical action trajectories as shared action embeddings, retrieves them with a pretrained action tokenizer, and conditions them on visual observations through cross-attention. A motion-aware transition loss is added to focus prediction on action-related visual change instead of background clutter. The authors report gains in simulation benchmarks and real-world cross-embodiment settings. ArXiv · AI/CL/LG's note
The authors say current WAMs predict visual dynamics and actions but lack an explicit way to carry prior action experience into new tasks. AED encodes historical physical action trajectories as shared action embeddings, retrieves them with a pretrained action tokenizer, and conditions them on visual observations through cross-attention. A motion-aware transition loss is added to focus prediction on action-related visual change instead of background clutter. The authors report gains in simulation benchmarks and real-world cross-embodiment settings. ArXiv · AI/CL/LG's note
score 4