Latent Graph Neural Networks for Human Skeleton-based Action Recognition

The exploration of spatial-temporal graph neural networks has been extensive, yet it remains a challenge due to the intricate variations in spatiotemporal dependencies across skeleton joint flows. In response, we propose a novel guidance mechanism within the latent 3D spatial-temporal space, termed Latent Graph Neural Networks (LGNN). Our approach involves designing a long-term spatio-temporal model with a high k-value to capture action information effectively within the latent spatial-temporal domain, thereby representing the spatiotemporal dynamics of skeleton sequences. To enhance feature information between distant nodes and address redundant information, we introduce a pair distance normalization operation in the latent space. This operation facilitates the formation of LGNN, where two streams of base LGNN seamlessly merge into a single network, enabling end-to-end training. Notably, our method significantly improves performance compared to state-of-the-art approaches and enhances accuracy across three widely used and challenging datasets for action recognition.