Ubiquitous BlowClick: Non-verbal Vocal Input for Confirmation with Hand-held Mobile Devices in the Field

Mobile devices have become integral to modern life, with tasks often requiring one-handed interaction. However, conventional methods, such as tapping, face limitations, especially in scenarios where hands-free operation is crucial, like driving or using AR glasses. Alternative approaches, including speech commands and non-verbal vocal input (NVVI), have been explored to address these challenges. While speech commands suffer from inherent delays, NVVI, characterized by simple audio signatures, presents a promising solution. This study aims to integrate and evaluate NVVI on mobile devices compared to traditional tapping interaction in a field study. Participants conduct the study in uncontrolled environments using their own devices. Therefore, we use machine learning-based NVVI classification on an Android-based architecture, providing a fast and resource-efficient pipeline. In the empirical evaluation, we assess the feasibility and effectiveness of the interface utilizing a reaction time task and ISO 9241:411 Fitts’ Law selection tasks. Our findings indicate that while tapping exhibits a speed advantage in conventional use, blowing is consistently detected and executed with efficiency, resulting in significantly faster input, particularly when the hands are initially distant.