KannadaGPT: Tuning Language Model for Kannada Text Generation

This study describes an approach for optimizing a pretrained language model for Kannada text production. We leverage the GPT-2 architecture, which was first trained on a large Kannada corpus, to build a strong language model capable of comprehending and producing coherent Kannada text. The model is further refined using a dataset including pairs of prompts and their accompanying completions, allowing it to provide contextually appropriate and linguistically precise replies. The process is divided into two stages: pre-training to establish a generic language understanding, and strategy fine-tuning to improve model output quality. Due to computing resource constraints, the model was trained across fewer epochs on limited datasets; nevertheless, the suggested technique provides a foundation that may be extended and refined to increase performance. The evaluation of the fine-tuned model shows considerable gains in generating meaningful and contextually suitable Kannada text, indicating its potential applicability in a variety of Kannada-specific Natural Language Processing (NLP) tasks. Attention maps visualization is also examined to highlight which words or tokens in a phrase have the most influence on predicting the following word in transformer-based models.