A New Human Action Recognition Method Based on Skeletons From Depth Sensors

Skeleton data from depth sensors, enriched with substantial information and compact in storage, provides a streamlined sequential representation. It is resilient to environmental variables such as background noise, attire, and illumination. Consequently, skeleton-based human action recognition emerges as an important topic in computer vision. Its key task is to extract distinctive features, specifically those critical for differentiating human actions. This work proposes an Ambiguous-feature-refinement and Orthogonal-basis-coordinate-transformation-based residual graph convolution neural network (AOnet) model for skeleton-based human action recognition. Firstly, multiple input branches are proposed and employed to enrich skeletal information features at the early data-input stage while maintaining a compact and effective representation of data. Secondly, since many actions exhibit highly similar joint-features, an ambiguous-feature-refinement model is introduced to further distinguish ambiguous features without considering external information interaction. This step benefits the extraction of distinctive spatial-temporal features, thereby improving the differentiation among action categories. Lastly, an orthogonal-basis-coordinate-transformation method is proposed to convert refined nonlinear spatial-temporal features into linear ones, and augment the model’s ability to capture higher-order dynamic information. The proposed method has been evaluated on two large-scale datasets. Experimental results confirm that AOnet demonstrates superior performance over the state of the art.