Real2Gym: Building Gyms from Videos, Bringing Skills to Robots
Real2Gym turns demonstration videos into editable simulation gyms, then uses them to transfer learned manipulation skills onto real robots.
The paper describes a Real2Sim2Real pipeline that reconstructs scenes from human or robot videos, aligns objects and cameras, and checks actions through physics execution. Inside those gyms, an agent writes and tests manipulation-stage code, keeping successful procedures, object-relative motions, and recovery strategies. The authors report better simulation success than GPT-6 Astra Direct Mode with far fewer policy-execution tokens, and higher physical execution success across four Franka robot tasks. HF Daily Papers' note
The paper describes a Real2Sim2Real pipeline that reconstructs scenes from human or robot videos, aligns objects and cameras, and checks actions through physics execution. Inside those gyms, an agent writes and tests manipulation-stage code, keeping successful procedures, object-relative motions, and recovery strategies. The authors report better simulation success than GPT-6 Astra Direct Mode with far fewer policy-execution tokens, and higher physical execution success across four Franka robot tasks. HF Daily Papers' note
score 5