Research of Multi-Stream Spatio-Temporal Graph Convolution Behavior Recognition Algorithm

In this paper, the convolution behavior recognition algorithm of multi-stream spatio-temporal graph based on Transformer fusion is proposed, and an innovative spatiotemporal self-attention unit is designed to model the long-distance dependence between all nodes in skeleton sequence, which reduces the dependence of behavior recognition algorithm on the accuracy of joint location, thus enhancing the robustness of behavior recognition algorithm. At the same time, the intermediate fusion strategy is adopted to realize the feature fusion between joints, bones and motion information in the skeleton sequence. The intermediate fusion allows the model to learn independent representations at all levels of information and then fuse them in the middle layer, making the model better adapt to the characteristics of each information. The accuracy of the algorithm on NTU-RGB+D60 X-Sub, NTU-RGB+D60 XView, NTU-RGB+D120 X-Sub and NTU-RGB+D120 X-Set are 91.2, 96.5, 86.3 and 87.9 respectively. Compared to a singlestream network, the parameters and floating-point operations are both reduced by <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$1 / 3$</tex>.