A Simulation and Training Platform for Remote-Sighted Assistance

Remote-sighted assistance (RSA) is a technology designed to provide assistance for visually impaired people (VIPs). In this scene, a remote-sighted agent communicates and sends commands to navigate and assist VIPs via real-time video sent back. However, the latency in real-time video and the deviation in the execution of instructions by VIPs are two important factors that affect the performance of agents to guide them. Therefore, how to enable agents to better guide VIPs under conditions of video transmission latency and deviation in instruction execution is an important issue. In this paper, we utilize Unreal Engine to create a virtual training platform for RSA, which simulates VIPs executing instructions in the real world and resembles the environment in RSA systems. We aim to help remote-sighted agents quickly master the set of vibration commands formed after encoding tactile vibrations and enable them to guide VIPs more effectively. Our experiment results show that, compared with untrained novices, when guiding people through the same path, agents trained on this platform reduce their average time by 32.09% and their average number of contacts with the environment by 57.57%. Our work provides agents with a simple and convenient simulation and training platform designed to enhance their performance by guiding VIPs with less travel time and fewer environmental contacts. Through this platform, agents can more effectively assist the visually impaired.