Revitalizing ancient languages through Deep Learning. A study on improving NLP accuracy of Vedic Sanskrit using Transfer Learning and Data Augmentation.

This study proposes a methodology for improving Natural Language Processing (NLP) of low-resource languages, specifically focusing on Sanskrit. The three-phase protocol includes Data Augmentation and Transfer Learning, Pre-training, and Evaluation/Benchmark comparison. In the first phase, word-to-word translations from Sanskrit to other languages are extracted from Panlex. A new vocabulary list is prepared by combining Sanskrit-English and Sanskrit-Hindi translation words, and the vocabulary of a base model is extended by adding these new tokens. The embeddings of corresponding words in different languages are aligned using a method called colocate_embeddings. In the second phase, after the alignment process, the model is further pre-trained on the target multi-class classification task using Vedic-sansksrit corpus to improve accuracy. Finally we train and evaluate the model's performance with different configurations and hyperparameters. In the final phase we've achieved nearly 38% increase in prediction accuracy with these techniques. The results indicate that transfer learning, when combined with multilingual models and cross-lingual alignment, can significantly improve the performance of models for languages that are less commonly spoken or have only recently been digitized.