LeapBoard: Integrating a Leap Motion Controller with a Physical Keyboard for Gesture-Enhanced Interactions

Abstract This research explores multimodal interaction whereby users engage multiple input and output channels, such as gestures, touch, audio, and graphics. Our system, LeapBoard, is a novel computer keyboard integrating a Leap Motion Controller (LMC) on a 3D printed frame. LeapBoard combines cursor positioning using mid-air gestures with selection using a key on a physical keyboard. In a user study with 12 participants using a 2D Fitts’ law target selection task, we compared LeapBoard with two alternative input methods, (i) a touchpad and (ii) an LMC combining mid-air gestures for cursor movement with a tap gesture for selection. LeapBoard yielded a significantly higher throughput (3.55 bps) than the touchpad (2.26 bps) and the LMC (1.97 bps). Error rates were significant lower for LeapBoard (4.6%) and the touchpad (3.4%) in comparison to LMC (12.8%). As well, the LMC method fared poorly on participant scores for fatigue (6.4/10). Fatigue scores were favorable for LeapBoard (1.8/10) and the touchpad (2.0/10). This work opens new possibilities for gestural and multimodal interaction while demonstrating off-loading of selection to alternate channels such as a separate hand or a dedicated key.