Development of a Mobile Robot With Vision-Based Real-Time Human Pose Estimation and Following Control

Outdoor 3-D human pose estimation (HPE) is critical for assessing motor functions of patients with degenerative diseases. To avoid wearable burdens and large footprints in daily use, a mobile robot is developed to enable contactless human-following control (HFC) and HPE in multiperson scenarios with the use of an represents red, green and blue (RGB) stereo camera. To address the reduction of the HPE accuracy caused by the short baseline of the stereo camera, observations of projected-limb orientations, 3-D ankle positions, and human frontal directions are designed to provide conditional high-confidence information. Combining the observations with kinematic priors, an HPE probabilistic model is formulated in a Bayesian framework, and the 3-D HPE is obtained by the maximum a posteriori (MAP). To provide feedback on the HFC, an explicit human-tracking algorithm in the sense of the MAP is designed to enable real-time HPE in multiperson scenarios. Based on the motion intent predictions from the HPE, a control algorithm with Lyapunov stability is designed to realize HFC of the robot with large-curvature turning motions. The experimental results show the accuracy of the HPE and stability of the HFC with mean errors of 49.88 mm and 21.1 cm, respectively.