Research on Action Recognition Algorithm Based on Multimodal Data Fusion

Along with the rapid development of AI and deep learning, action recognition has been widely applied in many areas such as safety monitoring, health monitoring, and so on. There are some limitations to the traditional single-mode action recognition in the face of complicated situations. To solve this problem, a novel action recognition algorithm (MM-CNN-LSTM) is presented in this paper. Through the combination of 3D-CNN and LSTM, the proposed algorithm is able to extract temporal and temporal features efficiently, and enhance the precision and robustness of action recognition. Through experimental verification, the NTU RGB+D dataset and the UCI HAR dataset were used for testing. The results indicate that MM-CNN-LSTM is better than the conventional single mode and other multimodal fusion methods. Specifically, the accuracy of the MM-CNN-LSTM model has increased by more than 15%, and the F1-score has also improved, which proves the advantages of multimodal data fusion in complex action recognition. In addition, the experimental analysis also shows that the fusion of video data and sensor data plays an important role in improving recognition accuracy.