Augmented Reality-Based Gesture Classification: A Data-Driven Analysis Using Meta Quest 3 Hand Tracking

This feasibility study investigates the use of standalone augmented reality (AR) headsets for gesture classification as a foundational component of sign language translation systems. A dataset of five distinct hand gestures was collected using the Meta Quest 3 headset, capturing 3D point rotations via the WebXR API in both single-user and multi-user settings. In the single-user setting, 100 recordings were captured for each of the five gestures. In the multi-user setting, gesture data was collected from 20 individuals, each performing five gestures with five repetitions. A complete processing pipeline was implemented for data preprocessing, labeling, and analysis. Feature selection combined Principal Component Analysis (PCA) with importance scores from a Random Forest (RF) classifier. Gesture classification was performed using standard machine learning models, namely RF and Support Vector Classification (SVC). In the single-user setting, RF achieved 99.26% accuracy and SVC reached 98.51%. In the multi-user setting, RF achieved 95.90% and SVC 95.55%. These results demonstrate that AR-based hand tracking offers strong potential for robust, contactless gesture recognition. Nevertheless, further research is required to evaluate its suitability for real-world assistive communication technologies.