Vision as Proprioception: Estimating Humanoid Robot Pose from Video
HumanoidPose cut joint-angle error by 18–32% against the strongest baseline on each of three real humanoid platforms.
The framework recovers full-body joint configurations in world space from video alone, without access to onboard state data. It combines pretrained human-pose features with robot kinematics, using synthetic training motions retargeted to 16 humanoid designs. Unlabeled online videos support self-training of a 2D robot-pose detector to improve real-world generalization. The authors also report better global localization and generalization to unseen robot designs.
HF Daily Papers' note
The framework recovers full-body joint configurations in world space from video alone, without access to onboard state data. It combines pretrained human-pose features with robot kinematics, using synthetic training motions retargeted to 16 humanoid designs. Unlabeled online videos support self-training of a 2D robot-pose detector to improve real-world generalization. The authors also report better global localization and generalization to unseen robot designs.
HF Daily Papers' note
score 5