Development of a Dynamic Event Hand Tracking System in Mobile Augmented Reality Viewers

This study introduces a dynamic event-handling system designed for hand gesture recognition in mobile augmented reality (AR) viewers. By repurposing low-cost virtual reality (VR) boxes in conjunction with mobile phones, the researchers aimed to facilitate hands-free interaction in AR environments. The system was developed using the Unity 3D game engine and integrated with ManoMotion for real-time gesture recognition, enabling the detection of ten distinct gestures, including Click, Drop, Pick, Grab, Swipe, and Release. Each gesture was mapped to dynamic events that allowed users to engage with virtual objects through natural hand movements, significantly enhancing the AR experience without reliance on physical controllers. The research confirms the viability of an affordable AR system that leverages intuitive hand-based controls while identifying limitations such as reduced gesture accuracy in low-light conditions and ergonomic challenges. The system achieved an impressive hand recognition accuracy of 95.13%. This study contributes to the accessibility of AR technology and lays the groundwork for future enhancements in gesture recognition and user interaction within mobile AR systems.