Transformers for Document Text Processing with a Focus on Complex Telugu Scripts

Recent advancements in the Optical Character Recognition (OCR) and Handwritten Text Recognition (HTR) domains have introduced various methods, those producing promising results. Traditional models, which includes Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and CNN-LSTMs, have been state-of-the-art models for HTR tasks for a long time. However, these models face problems when applied to complex scripts like Telugu and other Indian languages, which features intricate character shapes and extensive diacritics that are less captured by conventional and base state-of-art HTR methods. Further Transformer architectures, coupled with advanced augmentation techniques, and normalization techniques, have demonstrated significant improvements in the performance and accuracy over state-of-art models. These modern architectures, although yet to be completely applied to the scripts like Telugu and other Indian languages, has depicted promise in capturing the complex dependencies of handwritten text. The integration of advanced techniques like few-shot learning mechanism and self-supervised learning mechanisms also further increases the ability for accurate and robust HTR systems for complex script. When these methods are combined with Transformer-based architectures, have outperformed the traditional models on standard datasets. This review over such methods aims to provide an explanation of these methods, their experimental outcomes, and also their implications for the future of Handwritten Text Recognition (HTR) domain and document text processing, particularly for the complex scripts like Telugu.