Seeing Eye Stretch: Robot-assisted Indoor Navigation for Blind or Low-Vision Individuals

Most people with visual disabilities cannot use traditional assistive tools due to issues with accessibility, availability, and cost, highlighting the need for alternative inperson solutions. This work describes the conceptualization and implementation of an end-to-end Robot-Assisted Indoor Navigation (RAIN) system that can provide relevant feedback at appropriate moments to blind or low-vision (BLV) individuals as they move through indoor public spaces. Such a system needs to include multiple inter-connected modules: an Object Detection module that discerns inputs to establish the state of the world ahead while considering the user's customizable preferences, an Adaptive Navigation module that uses information from sensors to plan a path while avoiding collisions safely, and a Visual Question Answering (VQA) module that takes multimodal input (text and image), communicating with the BLV individual through a more natural interface that can process questions while maintaining alignment with the user's intent and awareness. Using this modular approach, we aim to inform both our system design and general guidelines for such assistive robot systems.