Hybrid Approach for Word Recognition in Bilingual Natural Scene Images

A multilingual country like India, where Kannada and English are the languages derived from the Brahmi and Latin scripts, respectively. In the Indian context, there is a need for Bilingual, Trilingual, and Multilingual word recognition in natural scene images to meet the requirements of a multilingual OCR system. Hence, a novel hybrid-based approach was proposed for extracting the features, such as the Resnet50 architecture in deep learning with 50 layers and utilizing residual learning through skip connections to enable efficient training of very deep networks. A total feature set of size 2048, after implementation of PCA, is reduced to 60 potential features. A dataset of 12,082 real-world sample images is collected, with diverse scenarios where bilingual text appears in various orientations, fonts, and complex backgrounds. In this paper, two experimental setups are carried out: a hybrid-based approach without PCA (Principal Component Analysis) and with PCA. The recognition accuracy was 97.48% using an SVM (Support Vector Machine) classifier without PCA and 97.56% with PCA, respectively. To test the performance of the Resnet50 model, a comparison is made with other pre-trained models like Vgg16, Google Net, Mobile Net, Efficient Net, and Vision Transformer, and later selected an optimum kernel like RBF with an SVM classifier to demonstrate the efficiency of the models. The novelty of this paper is the dimensionality reduction of weak features. The Time complexity of the SVM classifier with training and testing is reduced from 22% to 10% and it also demonstrates the capability of deep learning models to handle the complexities of bilingual text word recognition. It provides an effective solution in a multilingual environment.