Citizen Cane - An Object Detection and Image Description System for the Visually Impaired

Visually impaired persons can use informed visual perception to generate understanding of the environment around them to bridge the gap between them and world. This can be done by helping those individuals to identify visual features with the help of linguistic expression through aural/vocal or haptic means. Images are one example of visual perception; humans can identify images, describe it with their words (linguistic expressions) through speech (vocal). The proposed system provides the visually impaired persons with a Smart Cane, that provides the same haptic feedback for object detection, and Smart Glasses that capture the surroundings in the form of images that are captioned based on Encoder-Decoder using deep learning algorithms- Resnet101 which recognizes objects in the images and the LSTM (Long Short-Term Memory) which generates the image caption. The proposed model helps describe the image's features in a meaningful sentence with good accuracy and efficiency.