RO'YA: Machine Learning–Based Visual Assistance System

People with visual impairments are more likely to face serious life challenges compared to the general population, with 68% experiencing events like accidents, fires, or assaults, compared to 60% of the general population. This highlights the urgent need for innovative solutions to improve their safety and independence. Our project is a camera that can be fixed on a pair of glasses. This camera integrates advanced computer vision technologies, MiDaS for depth estimation, and YOLOv5 for object detection, to help visually impaired individuals navigate their surroundings. MiDaS calculates object distances, while YOLOv5 identifies and classifies objects. Real-time spoken feedback through text-to-speech informs users about their environment. The project’s app also allows users to customize objects for detection, such as keys, wallets, or family members by using face_recognition model, enhancing its relevance to daily life. To set up the system, a laptop equipped with a camera was used to capture live video input, processed via OpenCV. Depth estimation was performed using the MiDaS model, while YOLOv5 handled object detection. Pre-trained models were loaded with PyTorch, and the outputs were integrated to display annotated frames showing detected objects with their estimated distances. Testing was conducted in various environments to evaluate accuracy and usability. In testing, the system successfully detected a person at 0.99m from a controlled distance of 1m, verified using a meter ruler, demonstrating its accuracy. Potential sources of error include environmental factors such as lighting changes or slight movements by the user due to using a low-quality camera. However, these did not significantly impact the overall accuracy of the system. This project aims to enable blind users to navigate confidently in various environments, enhancing their independence and connection to the world thus reducing the accidents of visually impaired individuals. Future improvements include AI models for better object detection and hazard prediction, an AI assistant for locating lost items, multilingual audio feedback, and further customization options. Our vision is to empower every individual, regardless of their sight, to explore and thrive with confidence.