EgoX: Egocentric Cross-Embodiment Manipulation with Embodiment Dreaming
Pretraining on mixed human, robot and simulated demonstrations raised average normalized task progress from 0.63 to 0.80.
EgoX was evaluated across 13 task–embodiment pairs covering five tasks and six robot embodiments. Under out-of-distribution conditions, average task progress rose from 0.53 to 0.77. Its model shares a Transformer across embodiments while using dedicated interfaces for observation and action modalities. A training-only “embodiment dreaming” objective predicts future visual and proprioceptive representations; ablations found benefits from this objective, modular interfaces and broader pretraining mixtures.
HF Daily Papers' note
EgoX was evaluated across 13 task–embodiment pairs covering five tasks and six robot embodiments. Under out-of-distribution conditions, average task progress rose from 0.53 to 0.77. Its model shares a Transformer across embodiments while using dedicated interfaces for observation and action modalities. A training-only “embodiment dreaming” objective predicts future visual and proprioceptive representations; ablations found benefits from this objective, modular interfaces and broader pretraining mixtures.
HF Daily Papers' note
score 5