Motion-Based Human Spatial Problem-Solving Assessment Using Hand Movement Analysis

Spatial problem-solving is a cognitive process involving the identification, analysis, and resolution of challenges through mental strategies and critical thinking. This skill is essential for manipulating objects and solving complex spatial tasks. Insufficient problem-solving ability can lead to difficulties in decision-making, adaptability, and professional growth in spatial reasoning. This study aims to develop an index correlating with Mental Rotation Test (MRT) scores, inspired by traditional assessments and hand movement tracking data. Twenty-six participants took part in this study and initially completed a paper-based MRT. They then viewed a 3D shape from three perspectives on a monitor and constructed a matching model using cubes, while a camera-based real-time hand tracking system recorded hand key point coordinates. Predictors were developed by analyzing hand movement distance and time-based measures during the tasks. The proposed index achieved an R<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> score of 0. 7 and a mean absolute error of 2.32. This dynamic assessment method can potentially provide a faster, more cost-efficient, and detailed evaluation of human spatial problem-solving skills.