Reflex: Visually Grounded Reactive Humanoid Control
Reflex trains a humanoid to catch thrown boxes from egocentric RGB-D vision, then tests it on a real Unitree G1.
The system breaks the problem into whole-body catching, inference of box dynamics from partial delayed observations, and visual recovery of that dynamics representation. In simulation, it reaches 85.7% catch success from egocentric RGB-D, close to 87.9% with privileged box observations. Removing temporal visual history sharply hurts performance. On real human-thrown boxes, it succeeds 65% at 2.5 m and 40% at 5 m. HF Daily Papers' note
The system breaks the problem into whole-body catching, inference of box dynamics from partial delayed observations, and visual recovery of that dynamics representation. In simulation, it reaches 85.7% catch success from egocentric RGB-D, close to 87.9% with privileged box observations. Removing temporal visual history sharply hurts performance. On real human-thrown boxes, it succeeds 65% at 2.5 m and 40% at 5 m. HF Daily Papers' note
score 5