AI-Driven Comic Narration System for Visually Impaired Users

Comics, with their complex combination of visual storytelling and text-based dialogue, pose substantial accessibility issues for those with visual impairments. Current screen readers and accessibility solutions cannot handle the nonlinear and image-dependent nature of comic book content. The aim of this project is to design an automated system that provides individuals with visual impairments the opportunity to enjoy comic books through auditory storytelling. The system leverages a blend of computer vision, optical character recognition (OCR), and natural language processing (NLP) technologies to capture important details from comic book illustrations. This includes interpreting scene descriptions and text within speech bubbles. This information is compiled into a coherent narrative and transformed into speech using text-tospeech (TTS) technology. YOLOv11 is used for panel segmentation, ensuring the correct order of content is captured. After the segmentation, it analyses images, extracting text from speech bubbles, and providing detailed descriptions of scenes and characters, it provides a complete storytelling experience. This groundbreaking system offers a novel approach to making visual media, like comics, accessible for individuals with visual impairments by converting visual elements into an auditory format.