Computer-Vision-Enabled Worker Video Analysis for Motion Amount Quantification

The performance of physical workers is significantly influenced by the extent and quality of their motions. However, accurately measuring and assessing these motions remains a challenge due to the limitations in conventional instrumentation; wearable sensors require calibration and restrict mobility, while marker-based motion capture systems are costly and impractical for field deployment. Recent advancements have enabled in situ video analysis for the real-time observation of worker behaviors. To address these measurement constraints, this paper introduces a novel framework for tracking and quantifying upper and lower limb motions, issuing alerts when critical thresholds are reached. Using joint position data from posture estimation, the framework employs Hotelling’s T2 statistic to quantify and monitor motion amounts. A significant positive correlation was noted between motion warnings and the overall NASA Task Load Index (TLX) workload rating (r = 0.218, p < 0.005). A supervised Random Forest model trained on the collected motion data was benchmarked across multiple datasets, including the in-house assembly dataset, G-AI-HMS, UCF Sports Action, UCF50, and PE-USGC. The proposed framework identified motion anomaly patterns with a maximum accuracy of 94% on the in-house assembly dataset, while performance varied across the external benchmark datasets.