H2E : Hand, Head, Eye a Multimodal Cascade of Natural Inputs

Eye-based interaction techniques for extended reality, such as gaze and pinch, are simple to use however suffer from input precision issues. We present H2E, an integrated fine and coarse-grained pointing framework that cascades Hand, Head, and Eye inputs. We further introduce the MagicPinch gesture as an example of the framework, which allows for smooth transitioning between fine and coarse-grained pointing. When combined together, after users initiate a pinch gesture, a cursor appears midway during the pinch at the position of the gaze, which can be dragged by head pointing if needed before pinch confirmation. This has the advantage that it can add a precision component without changing the semantics of the technique. In this paper, we describe the design of the H2E framework and implementation of the MagicPinch technique. Furthermore, we present an evaluation of our method in a Fitts-based user study, exploring the speed-accuracy trade-offs against a gaze and pinch interaction baseline.