Enable Natural User Interactions in Handheld Mobile Augmented Reality through Image Computing

Augmented Reality (AR) has transformed user interaction with the digital world by blending virtual elements with the physical environment. While various AR interaction methods have been studied, most research has concentrated on evaluating interactions within controlled experimental settings, lacking validation in practical application scenarios. This project aims to address this gap by using Unity to develop a handheld mobile AR application that simulates the pottery-making process. Two versions of the application were developed: a conventional touch-based version and a mid-air natural interaction version based on hand detection and tracking. We conducted a user study of 40 participants adopting the mixed methods design. By comparing the data from the touch-based interaction group (N=20) to the mid-air natural interaction group (N=20), the study found that the touch-based version had better usability, while the mid-air version provided a more immersive experience, as acknowledged by the participants. Future work should focus on addressing areas for improvement in the mid-air interaction, which could include incorporating better distance indications and interaction feedback, thus further bridging the gap between virtual and real-world interactions.