Manifold Matrices-based Attention Mechanisms on 3D Skeletons for Human Action Recognition

Currently, one of the most well-liked study fields in computer-vision is skeleton-based human action recognition. As the Lie group in Riemannian manifolds is able to precisely describe 3D geometric relationships among rigid bodies, it is widely used in skeleton-based action recognition approaches to construct action feature descriptors. Regrettably, the majority of these approaches overlook crucial body parts and skeletons in favor of focusing solely on spatio-temporal descriptors of an action as a whole. A manifold-based rigid body motion attention mechanism is proposed to assign varying degrees of importance to the relative geometries of different limb motions, and then a skeleton-based spatial attention module is constructed on the basis of limb motions for the more efficient extraction of spatial features from skeletons. Furthermore, a Lie group-based temporal attention mechanism is proposed on the basis of the first two phases to choose significant skeleton frames in an effort to raise the level of action recognition accuracy even higher. Results from experiments on four of the most influential action datasets demonstrate that the proposed approach performs better in terms of action recognition accuracy than many cutting-edge approaches based on skeletons.