An AIoT-based framework for human activity recognition and performance assessment

Abstract The ability to consistently improve human performance for physical activities in work or sports depends on the trainer’s assessment. Most current assessment techniques include manual observation and are often not kinematic data-driven to provide accurate and personalized performance assessments. To overcome such limitations, researchers are exploring Internet of Things (IoT) sensors to accurately and comprehensively measure the human body movements in physical activities and using various Artificial Intelligence (AI) models to assess performance. Despite the progress in AI and IoT (AIoT)-based approaches, the limitations remain due to a lack of data from real-world tasks and environments. To address these limitations, this paper proposes a novel framework, PerfoMax-AI, which assesses performance by combining full-body IoT sensing with an AI model for activity recognition and performance assessment. This framework uses an AI model referred to as a hybrid GAN-ML model, which combines a Generative Adversarial Network (GAN) for synthetic data generation, Synthetic Minority Over-sampling Technique (SMOTE) to mitigate the class imbalance in real-world datasets, an Edited Nearest Neighbors (ENN) model for noise reduction, and a Machine Learning (ML) model for activity recognition and performance assessment. To evaluate the proposed framework, a real-world use case, sports climbing, was used. The framework achieved 98% accuracy for activity recognition and 92% for performance assessment on the training dataset. Evaluation of unseen participants achieved 86% accuracy for activity recognition and 84% for performance assessment, indicating its potential for real-world sports climbing performance assessment.