AgentHands: Generating interactive hand gestures for spatially grounded agent conversations in XR
Google’s XR prototype turns an assistant’s spoken instructions into timed hand motions anchored to real objects.
AgentHands uses gaze, scene reconstruction, and a gesture-event library so virtual hands can point, trace, warn, or demonstrate actions in the user’s space. In tests with 12 participants, Google compared it with speech-only guidance for orchid care and 3D printer tasks. The gestural version scored better for finding objects, following actions, noticing warnings, and remembering instructions. Source: Google Research's note
AgentHands uses gaze, scene reconstruction, and a gesture-event library so virtual hands can point, trace, warn, or demonstrate actions in the user’s space. In tests with 12 participants, Google compared it with speech-only guidance for orchid care and 3D printer tasks. The gestural version scored better for finding objects, following actions, noticing warnings, and remembering instructions. Source: Google Research's note
score 4