A Novel Approach for Leveraging Object Detection for 3D Human Pose Estimation in Complex Human-Robot Collaboration Environments

Although 3D Human Pose Estimation has major breakthroughs in recent years, 3D pose estimation in complex scenarios remains difficult. One of the reasons is the lack of diverse 3D datasets for training and generalizing the models. This issue is counteracted by acquiring a dataset of Human-Robot Collaboration scenes featuring different objects, such as a cobot. We propose a novel two-step method, where first a 3D Object detection task with VoteNet is performed to identify the human in the scenario and claim it as a region of interest for the pose estimation task. Second, this region of interest is cropped and passed into the 3D Human Pose Estimation algorithm SPiKE, which locates 15 keypoints of the human. Based on this procedure, our method improves detection in complex scenarios. Furthermore, this article compares the benefits of training the algorithm additionally on the obtained Human-Robot Collaboration dataset compared to training it with the standard ITOP dataset. While the SPiKE algorithm makes no correct prediction on the Human-Robot Collaboration scenario, the results of the two-step SPiKE<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">VN</inf> approach with mAP of 41.17 % is significantly lower as the benchmark model on the ITOP dataset. Nonetheless, the SPiKE<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">VN</inf> model exhibits similar performance to SPiKE<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">man</inf> with a difference of 2.43 % mAP indicating the method is effectively functioning.