Infrastructure Enabled Guided Navigation for Visually Impaired

Despite advancements in navigation-assistive technology, independent outdoor traveling remains challenging for individuals with vision loss due to uncertain information. We present an outdoor navigation assistive system that collaborates with infrastructure to address these limitations. Our system includes an RGB-D inertial sensor, GPS sensor, Jetson Orin NX 16GB, and bone-conduction headphones. It improves localization accuracy through semantic segmentation, depth map enhancement (Depth-Decoder), and a prior map. The proposed semantic segmentation convolutional neural network (CNN) is designed to operate on low-compute devices, achieving competitive performance with 71.2 mean Intersection of Union (mIoU) on the Cityscapes and 74.46 mIoU on the Camvid. It operates at 209 frames per second (fps) with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$480\times 848$ </tex-math></inline-formula> image resolution on the GTX 1080 desktop and 50 fps on the low-compute Jetson. Additionally, our Depth-Decoder enhances raw depth maps using geometrical loss with semantic segmentation constraints and structure-from-motion. Depth-Decoder demonstrated an average deviation of 0.66m compared to 2.23m from raw depth maps for landmarks within 10m. Notably, the outcomes of both CNNs are inferred from a single RGB image. Two visually impaired individuals assessed the technology. Participant S001 completed 6/8 trials and S002 completed 7/7 trials. Infrastructure collaboration is demonstrated by using existing prior maps and by dynamically adjusting paths based on real-time information from sensing nodes, to avoid collisions with oncoming agents not visible with the wearable camera. This research provides fundamental design principles for future outdoor navigation assistive technologies, offering insights into addressing challenges for people with visual impairment and other vulnerable road users.