Enhancing hand-object interactions in virtual reality for precision manual tasks

The realism and immersion of Virtual Reality (VR) experiences depend on the quality of interactions between the virtual hand and virtual objects. The current drawbacks, such as seemingly artificial hand-object interactions and erratic post-collision behaviors of both virtual objects and the virtual hand, curtail the effectiveness of VR in tasks requiring precise manipulation, precluding sustained and successful adoption of VR in precision manual tasks. To address these limitations, we advocate a strategic approach across three related domains: (1) Developing a sufficiently realistic virtual hand model (i.e., a set of rigid bodies or deformable meshes) that can implement the complex movements of a biological hand. (2) Exploiting synergistic patterns of multi-digit motion and contact forces revealed by research on neuroscience, psychophysics, and manual actions to develop hand-object collision handling algorithms. (3) Implementing seamless and fluid releases of whole-hand virtual grasps, especially involving complex grasps and in-hand manipulation tasks. This article explores various aspects of virtual grasping that go beyond traditional physics simulations, proposing innovative solutions to overcome technical barriers across these domains.