Fusion of Structural and Temporal Features for Human Action Recognition

Human Action Recognition(HAR) is a pattern recognition task that identify specific human actions or motions. Since the amount of video content being uploaded to websites are increasing, the extraction of useful information from these videos are of great importance. The applications of HAR includes: automated surveillance, sports video analysis, gesture recognition etc. Earlier techniques primarily relied on either local features or temporally extracted features, limiting their effectiveness in capturing the complete motion dynamics. Hence a method is to be proposed that uses the composition of both temporal and structural features to provide better identification rate. The action type is identified from the pose sequence in the video. The various action types used includes walk, sit down, pick up, running etc. For each joint, an angle is to be calculated between each joint to obtain structural features. When individual joints are observed from one frame to another, the dynamics of the joints are obtained and hence temporal features can be extracted. Therefore structural features and the temporal features can be fused to see whether any increase in the classification accuracy can be made. The neural network can be used to fuse the features to perform classification. The experiment can be conducted on several datasets such as KTH, MSRAction3D etc to verify how these accuracy differs among each.