A multimodal Lidar system for auditory spatial awareness of visually impaired individuals in urban environment

Visually impaired individuals face significant challenges in navigating and interacting with their environment due to partial or total loss of sight, which can result from congenital conditions, illness, injury, or age-related degeneration. This sensory limitation often impacts mobility, independence, and access to information, making assistive technologies and adaptive strategies essential for improving quality of life. Solutions such as tactile aids, auditory feedback systems, and advanced tools like computer vision, GPS-based navigation, and sonification are increasingly being developed to enhance spatial awareness and safety, enabling visually impaired persons to engage more confidently with the world around them. This study presents a novel lidar-based system that converts spatial information into a binaural audio representation, enabling visually impaired individuals to perceive their environment through sound. As proof of concept, our system uses a small lidar sensor controlled by two servo motors to capture distance information, which is then translated into a color-coded image. This image is subsequently transformed into a 3D audio soundscape, where distances and spatial locations are encoded as distinct musical instruments forming an orchestra in front of the subject. The spatial audio uses ambisonic concept to a more immersive experience and converted to binaural sound for headphones listening. An integrated 2D distance image sensor, designed to measure the distance to an object by TOF method will be tested to optimize speed and reliability of the image acquisition process. Key Features of the system are the accurate distance information and its color-coded image representation, providing visual-to-audio mapping by using binaural able to provide Real-time audio feedback, enabling navigation and interaction with surrounding urban environment. This system has the potential to significantly enhance spatial awareness and mobility for visually impaired individuals. By implementing multimodal processing and sensory substitution, this system can provide an intuitive and immersive experience, ultimately improving users' quality of life. Future studies will focus on user testing, system refinement, and exploring applications in various environments and scenarios.