TSN vs I3D vs Pose-C3D: Action Recognition in Basketball using SpaceJam Dataset

Most of the research conducted in action recognition is mainly focused on general human action recognition, and most of the available datasets support studies in general human action recognition. In more specific contexts, such as basketball, datasets that are as comprehensive and publicly available are limited. This study proposes taking three popular and mature methods in the field of action recognition, namely Temporal Segment Networks (TSN), Two-Stream CNN using Inflated 3D-convolutional Neural Networks (I3D) and Pose-C3D, and applying them to the SpaceJam dataset, which is a basketball-specific action dataset. All three experiments used pre-trained ImageNet models and were fine-tuned on the SpaceJam dataset. TSN was the oldest of the methods but obtained the best results of the three experiments, scoring a top-1 and top-5 accuracy of 59% and 96%, respectively. I3D was second best, with a top-1 and top-5 accuracy of 41% and 85%, respectively. Pose-C3D came in third, scoring a top-1 and top-5 accuracy of 15% and 50%, respectively. The results show that the models cannot distinguish significantly between some actions, such as ball in hand, pass and dribble. The study shows that it is feasible for context-specific fine-grain action recognition, but more needs to be done to discriminate against similar actions.