Automated and efficient Bangla signboard detection, text extraction, and novel categorization method for underrepresented languages in smart cities

Developing cities face a critical challenge in ensuring equitable access to urban information, as millions of establishments remain invisible on digital platforms due to the labor-intensive and inefficient process of manual labeling and categorization. Accurate signboard detection, labeling, and categorization in natural scenes are essential for cities to become truly inclusive and connected smart ecosystems. This foundational task is essential to enabling transformative solutions like translation tools for non-native speakers, accessibility aids for visually impaired people, and comprehensive urban data platforms. The development of such automated systems faces significant challenges due to variable lighting, intricate backgrounds, fonts, designs, and varying name patterns. This study presents an efficient End-to-End Automated Bangla Signboard Detection, Text Extraction, and novel Categorization System optimized for edge devices. The proposed system is structured in four integral phases. The model first detects signboards from input image, then identifies Regions of Text Interest (RTI) containing names and addresses. This is achieved using the YOLOv9-s model, optimized with custom learning rate adaptation for enhanced precision and efficiency. In the third stage, the Tesseract OCR, enhanced with custom image preprocessing, is used for text extraction. Finally, fine-tuning of the pre-trained multilingual Bidirectional Encoder Representations from Transformers (BERT) model is implemented to mimic human perception to achieve Named Entity Recognition (NER) capabilities.These models are trained on the most extensive collection of over 16,402 Bangla signboard images, along with 42,548 categorized names of establishments curated by us. The accuracy achieved in the four phases is 99.1%, 92.89%, 91%, and 92.89% respectively. • A real-time system to detect Bangla signboards, localize RTI, and extract text. • A Name-Entity Recognition system classifies establishments from extracted names. • Quantization is applied to YOLOv9-s for Bangla signboard detection on edge devices. • A statistical method enhances OCR accuracy through image binarization techniques. • Auto-learning rate adaptation for YOLOv9-s improves training resource efficiency.