3D Skeleton-Based Sport Action Recognition System via Deep Learning

In this study a system based on deep learning was developed to recognize 5 sports movements (kick, straight punch, hook, uppercut, serve). In the first stage, an application was developed for the simultaneous capture of Red Green Blue (RGB) video frames and three-dimensional (3D) skeleton data using a specialized camera. The developed system includes a user-friendly interface for data collection and video recording editing. To identify these sports movements, videos were recorded from ten different individuals, from 5 different angles, with 2 repetitions each, creating a custom dataset containing 500 samples. To characterize the temporal dependencies in the skeletal data in this dataset, 1D-CNN (One-Dimensional Convolutional Neural Network) and LSTM (Long Short-Term Memory) deep learning models were used, and the effectiveness of combining these models was investigated. The best-performing model was determined through cross-validation. Different activation functions and optimizer combinations were analyzed in detail for this model. With the best-performing model, a classification accuracy exceeding %87 was achieved for the 5 sports movements.