Enhancing Automated Feedback for Fitness Exercises Using Pose Estimation and Semi-Supervised Generative Adversarial Learning

In the context of unsupervised home workouts, the provision of automated feedback is imperative to mitigate the risk of injuries. To date, the literature has not addressed the challenge of video input from a single smartphone camera, which is hindered by the presence of varying perspectives and occlusions. We extend existing approaches based on three-dimensional (3D) pose estimation and Graph Convolutional Networks (GCNs) with semi-supervised Generative Adversarial Networks (GANs) to robustly work with the noisy 3D coordinate approximations inferred from single-camera inputs. The SmartFit dataset has been developed to model variations in inputs in a natural manner by enabling test subjects to record themselves in arbitrary positions using their smartphones. It is only partly labeled to show the ease of extending our method with little labeled data. The efficacy of the semi-supervised model on the SmartFit dataset has been demonstrated by an 11.4 % higher mistake classification accuracy compared to the original, fully-supervised model.