LookUP: Command Search Using Dwell-free Eye Typing in Mixed Reality

We introduce LookUP, a novel general purpose command search system for mixed reality headsets, offering a hands-free experience through dwell-free eye typing. With LookUP, users can trigger the display of a virtual keyboard with a simple upward head motion. The keyboard then uses a statistical decoder to interpret users’ intended text based on their eye movements. This approach diverges from traditional dwell-time methods, significantly enhancing typing speed and efficiency. Our research involved deploying LookUP on a HoloLens 2, and benchmarking it against a dwell-based command search baseline and the native HoloLens system menu. Our user study indicated that participants spent a significantly shorter time using LookUP with dwell-free eye typing in command search and entry, demonstrating LookUP’s potential to be a complementary command input for mixed reality headsets.