Object center of mass predicts endpoints for free-hand pointing in virtual reality

Humans learn to point using their index finger at a very young age, intuitively directing other people’s attention to objects in the environment. Typically, this is achieved by bringing the dominant eye, index finger tip of the dominant hand, and pointing target into alignment (eye-hand alignment; also termed "eye-finger-raycast" when used in human-computer interaction). This requires the human sensorimotor system to select an appropriate target location to plan the pointing movement. While free-hand pointing gestures have been extensively studied using small and well defined targets (e.g., numbers shown on a wall), it is currently less understood how humans select a specific target location on a more complex three-dimensional (3D) object. Eye movement research suggests that the center of an object has the highest likelihood to be fixated, and that this fixation location shifts to an object’s implied 3D center of mass (CoM) if object characteristics such as shading imply a 3D shape, making the CoM a logical candidate. Here, we investigated whether the CoM also serves as a target for eye-hand alignment when pointing to 3D objects in virtual reality (VR). Participants pointed at custom objects ("vases") using their index finger, using different visual feedback and from different vantage points in a virtual room. Hand movements were tracked using a Vive Tracker. We computed pointing vectors for each trial using eye and hand tracking, and then estimated 3D endpoints within the object from averaged pointing vectors. Endpoints were best predicted by CoM, and manipulating 3D object shape to systematically shift the CoM induced corresponding shifts in participants’ pointing endpoints. The type of visual feedback provided during pointing influenced overall accuracy, but did not influence the effect of CoM manipulation. These results suggest that object CoM plays a major role in eye-hand alignment when pointing to 3D objects in VR.