RoboTok: An Internet-Scale Data Engine for Human Demonstration Retrieval and Dexterous Manipulation Learning
RoboTok turns web videos of human hands into searchable training data for dexterous robot policies.
The paper describes a retrieval engine that takes a human manipulation video as a query and searches internet-scale video collections for relevant demonstrations. Its core representation uses 3D hand trajectories in actor-centered reference frames, meant to compare actions across camera angles, scene appearance, and occlusion. The authors report stronger retrieval results than existing robot-data retrieval methods and improved downstream robot task success. Source: HF Daily Papers' note
The paper describes a retrieval engine that takes a human manipulation video as a query and searches internet-scale video collections for relevant demonstrations. Its core representation uses 3D hand trajectories in actor-centered reference frames, meant to compare actions across camera angles, scene appearance, and occlusion. The authors report stronger retrieval results than existing robot-data retrieval methods and improved downstream robot task success. Source: HF Daily Papers' note
score 5