Robot-assisted navigation with adaptive impedance and path planning for visually occluded users

Considering the number of visually impaired individuals worldwide and the limited availability of assistance for safe navigation, there is a growing need for intelligent robotic systems that can provide reliable guidance. This manuscript presents an adaptive 3D human guidance approach using a mobile manipulator to assist visually impaired users as they navigate unstructured spaces. The proposed system generates and executes a collision-free path toward a designated goal while avoiding both static and dynamic obstacles at varying heights for the entire system, which comprises of an omnidirectional mobile base, a torque-controlled robotic arm, and the physically coupled user interacting with the arm’s end-effector. To ensure the full-body safety of the user, we introduce a leg tracking algorithm based on 2D LiDAR sensors integrated into the mobile base, along with a human height estimation algorithm that uses an RGB-D camera. Based on the real-time tracking of the human state, the proposed approach provides responsive guidance: Horizontal Pulling adjusts the robot’s lateral motion and base velocity according to the user’s leg position when deviations from the collision-free trajectory occur, while Vertical Pulling modulates the arm’s height to prevent upper-body collisions, based on estimated user height and the obstacle information from an attached 3D LiDAR. Additionally, an Impedance Tuning algorithm is designed to dynamically adjust impedance parameters in coordination with the pulling force based on the estimated collision risk level. The effectiveness of the proposed approach is demonstrated through extensive multi-subject user studies in controlled environments and a proof-of-concept deployment in a real-world scenario.