Human action recognition method based on spatiotemporal decoupling multi-scale feature fusion network

Aiming at the problems of incomplete expression ability of a single neural network, ignoring the spatiotemporal characteristics of time series data and slow reasoning speed and low accuracy of the traditional network model in wearable behavior recognition tasks, this paper proposes a human behavior recognition method based on spatiotemporal decoupling multi-scale feature fusion network. Firstly, the Multi-Dimension Convolutional Neural Network (MDCNN)is used to extract the different scale features of multi-channel sensor data, and the shallow and deep features of the signal are extracted at the same time. The complementary information between various levels of features is fully utilized to extract the features that represent the action information. Then, through the Decoupling Temporal-Spatially Transformer(DTST) based on the Transformer, the network is decoupled into time decoupling and space decoupling structures, to accurately extract the spatiotemporal features in the inertial signal. Finally, the Softmax classifier is used for action classification and recognition. The experimental results show that the proposed method can effectively utilize the consistency and complementarity between multi-sensor data, fuse multi-scale spatiotemporal features, and improve the reasoning performance and recognition accuracy in human behavior recognition. The average accuracy rate on the UCI-MHEALTH dataset is 96.83 %, and the average F1 value is 96.81 %.