Multi-object Detection and Natural Language Alert System for Walking Safety of the Visually Impaired

According to the World Health Organization (WHO), 285 million people suffer from severe vision loss. Assisting the daily lives of visually impaired individuals has been a long-standing research topic. There is a high demand for support tools that allow visually impaired individuals to safely navigate to their destinations by providing information after exploring the surroundings. The walking safety assistance system developed in this paper aims to detect situations and obstacles that may occur in the walking scenarios of visually impaired individuals and generate appropriate natural language sentences to be delivered via voice. The multimodal approach processes and understands visual and language data simultaneously to provide accurate and efficient information. The multimodal AI walking assistance system for the visually impaired is designed to respond quickly and accurately to situations and obstacles encountered during walking. It constructs a pipeline using the object recognition model YOLO and the large-scale natural language model Koalpaca, providing convenience through a voice delivery system.