Multimodal Assessment of Expertise in AR-Guided Psychomotor Tasks

To enable personalized augmented reality (AR) systems that adapt in real time to a user’s level of expertise, it is first necessary to assess individual skill levels during task performance. In this article, we introduce a novel data-driven approach for the multidimensional assessment of expertise in AR-guided psychomotor tasks. We evaluate users’ performance against multiple indicators by employing pairwise correlation and principal component analysis (PCA) on multimodal data collected from AR devices and wearable sensors—including hand tracking, galvanic skin response (GSR), and gaze. The results show that objective measures—such as visual scanning efficiency, stress indicators, inspection accuracy, and hand dexterity—provide valuable insights into multiple indicators of expertise and skill acquisition. These findings validate the potential for AR systems to adapt dynamically to users’ expertise levels based on observable data. This approach offers a promising direction for enhancing the effectiveness of AR-based training systems in high-stakes domains such as manufacturing, aviation, and medicine, which can benefit from real-time, multimodal assessment of expertise for adaptive AR guidance and assistance.