Dual-Modal Human Pose Reconstruction Using 3-D UWB Radar Images and Point Clouds

This article proposes a multimodal human pose reconstruction method based on 3-D ultrawideband (UWB) radar images and point clouds, aiming to improve the accuracy of human pose estimation through the fusion of radar images and point cloud data. First, a UWB 3-D imaging radar system is designed, which synchronously collects radar echo signals and optical images, constructing a multimodal dataset covering various common actions and different human characteristics. Radar data processing includes azimuth-range 2-D imaging, target locking, local 3-D imaging, discrete sampling, and maximum projection to generate point cloud data and projection images. Optical image processing uses mature methods to reconstruct 3-D poses as pose labels for point clouds and projection images. To achieve multimodal data fusion, the UWB FusionPose network is designed, comprising an image feature extraction network, a point cloud feature extraction network, and a pose reconstruction network. The image feature extraction network is based on the ResNet-18 framework, while the point cloud feature extraction network adopts a pyramid structure. After feature fusion, a multilayer perceptron (MLP) is used to predict human pose information. Additionally, this article explores the impact of fusion parameters on network performance and verifies the effectiveness of the multimodal network through ablation experiments. Experimental results show that this method effectively utilizes radar point cloud data and projection image data to accurately reconstruct the 3-D pose of human targets. This research not only provides a new human pose reconstruction technique but also offers valuable references for the future development of radar imaging technology and multimodal data fusion methods.