Machine Learning-based Automatic Language Identification forIndian Languages

In multilingual civilizations like India, where many languages coexist, language identification is an essential problem in natural language processing. This study investigates the effectiveness of supervised learning techniques for language identification across a variety of Indian languages, including English, Hindi, Tamil, Punjabi, Kannada, Kangri, Assamese, Kokborok, Telugu, Marathi, Nepali, Malayalam, Bengali, and Oriya. These techniques include N-gram models, Naive Bayes (NB), Support Vector Machines, and Convolutional Neural Networks. The study makes use of a corpus that comprises representative text samples from every language and assesses how well the previously discussed techniques work in correctly categorizing text into the appropriate language. The results show that the various algorithms are not all that effective, with CNN showing promise in managing the intricacies of Indian languages. In addition, the study looks into how N-gram size affects classification accuracy and considers possible improvements to boost language recognition systems’ effectiveness in multilingual settings. Overall, by providing insights into efficient methods for automatic language identification across Indian languages, this research advances language technology in a variety of linguistic environments.