A Deep Learning-Based Scene Description and Object Recognition App for the Visually Impaired

The visually impaired face significant challenges in identifying objects and understanding their surroundings, often requiring assistance to navigate and interpret their environment. Current applications designed for blind individuals lack efficiency and present several limitations, including the absence of text-to-speech features and voice-controlled navigation. These limitations hinder the independence and mobility of blind users, complicating everyday tasks such as navigating inside a building and moving through public spaces. The proposed work aims to provide a comprehensive solution through a fully voice-controlled app utilizing deep learning to analyze and describe scenes and objects. By implementing TensorFlow Lite for optimized on-device processing, the app ensures efficient real-time analysis on mobile platforms. Leveraging an attention-based CNN-RNN model, the system performs feature extraction and generates detailed descriptions of scenes. Additionally, text-to-speech functionality offers immediate audible feedback, while voice commands enable hands-free operation. Extensive testing and optimization ensure a user-friendly and accessible experience, empowering blind users to independently interact with their environment and enhancing their autonomy and quality of life.