An Omnidirectional Guidance Robot System for Visually Impaired Individuals with Nonlinear MPC

In this article, we develop an intelligent guiding robot system based on omnidirectional kinesthetic feedback. This system provides a safer and more efficient solution for the independent travel of blind and visually impaired people (BVIP). The system includes a novel omnidirectional kinesthetic feedback guidance device and a platform equipped with various sensors (LiDAR, depth camera, IMU, etc.) and a computing unit. Meanwhile, we construct a perception-planning-control framework. To ensure the safety of the person during the guidance process, we develop a human-robot coupling model to describe the real-time state of the human-robot system. Tracking controllers execute these commands to drive the guiding device. Our system is deployed on the Smart Cane platform and has been validated through simulations and real-world scenarios, and demonstrate excellent trajectory tracking capability.