Multi-Lingual Handwritten Recognition Using Convolutional Recurrent Neural Networks

Character recognition from handwritten text, particularly for Tamil and English scripts poses special difficulty because of many styles of characters in this script, and due to the complexity of these scripts. In this paper, we introduce a robust convolutional recurrent neural network (CRNN) model to accommodate the vagaries of both Tamil and English handwritten character recognition. Using a comprehensive dataset with more than 60,000 images, the model includes English characters (0 to 56), as well as Tamil (0 to 155), capturing the characters of both languages exhaustively. The proposed architecture of the CRNN, includes a convolutional neural network for feature extracting and a long short-term memory network and the gated recurrent unit both of which allow for sequence understanding. The input images are then normalized and undergo several convolutional layers, succeeding LSTM and GRU layers which handle the sequential information that is important in character recognition. In the evaluation performance metric, our proposed CRNN model distinguishes Tamil script characters with 92% recognition rate, and the English script characters with 95% recognition rate. Furthermore, the designed model gets the average F1 score of 0.91 with the precision 0.93, and recall of 0.90 which improved than the traditional CNN based OCR systems and baseline CRNN architecture. These metrics demonstrate how well the model is able to detect intricate forms of characters in both script and how immune it is to variations in style. As a speech component to enhance the system, integrating Text to Speech (TTS) with gTTS technology turns the identified text into Tamil or English and makes our system user-friendly. Not only does the model provide a text form of what it recognized, it also produces the audio to have a full functional output and to expand the use of the model to audio processes. This paper demonstrates that the proposed deep learning models and the integration of TTS can enhance the recognition of bilingual handwritten characters in documents and promote document digitization applications, education, and language-related initiatives.