Human Action Recognition Based on Bimodal Data and Spationtemporal Features Using UWB MIMO Radar

Human action recognition (HAR) methods based on ultra-wideband (UWB) multiple-input multiple-output (MIMO) radar have demonstrated substantial potential in complex environments. However, the use of single-modal data, such as spectrograms or point clouds, poses significant limitations; it is easy to get confused when dealing with similar actions, particularly in challenging scenarios. To overcome these limitations, this study proposes an action recognition approach that leverages bimodal data and spatiotemporal features for improved performance. Four-dimensional radar image data of the human body are captured using UWB MIMO radar and processed into four-dimensional point cloud data and four-dimensional intensity projection image data, which collectively serve as bimodal feature inputs. To process these bimodal feature data, a network architecture named Point Image Fusion-Net (PIF-Net) is developed, incorporating spatiotemporal sequence classification to achieve recognition of human actions. The network initially extracts spatial features from the bimodal feature data, followed by temporal feature extraction, culminating in the classification of spatiotemporal features. Experimental results show that the proposed method achieves an accuracy of 96.75% in action recognition tasks, validating the approach.