Understanding Novice Users' Mental Models of Gesture Discoverability and Designing Effective Onboarding

A variety of consumer Augmented Reality (AR) applications have been released on mobile devices and novel immersive headsets over the last five years, creating a breadth of new AR-enabled experiences. However, these applications, particularly those designed for immersive headsets, require users to employ unfamiliar gestural input and adopt novel interaction paradigms. This leap forward intensifies the complexity of help-seeking and onboarding needs for the end-users. Recent emergence of artificial intelligence (AI)-powered in-context help tools has become potential alternatives to onboarding and search methods. However, non-technical users struggle with prompt-based interactions within LLMs that offer human-like language capabilities, which is unique, but can also be unreliable. My doctoral research aims to (1) understand how novice users discover gestural interactions and classify the types of interaction challenges they face; (2) investigate the nuances in users? mental models of emerging technologies, such as LLMs and AR; and, (3) explore the design of onboarding that enhances gesture discoverability and their application within the AR environments.