Ego Guidance: Enhancing Robot Policy Learning with Action-Free Human Egocentric Videos
Human first-person videos improved robot performance on unfamiliar tasks without action labels for those videos.
The researchers’ SPAR architecture builds on a pretrained vision-language model and learns to predict both actions and future visual states. They report that visual-state prediction supplies learning signals that complement action prediction. Representations learned from egocentric videos improved performance on out-of-distribution robot tasks, with further gains as the video data increased. The abstract gives no numerical results.
HF Daily Papers' note
The researchers’ SPAR architecture builds on a pretrained vision-language model and learns to predict both actions and future visual states. They report that visual-state prediction supplies learning signals that complement action prediction. Representations learned from egocentric videos improved performance on out-of-distribution robot tasks, with further gains as the video data increased. The abstract gives no numerical results.
HF Daily Papers' note
score 5