SentiGrad: A New Hindi-English Code Mixed Sentiment Analysis Dataset with Preliminary Results and Open Challenges

Sentiment analysis of the users' comments on social media enables educational content developers a quick and effective way to understand viewers' feelings about the content. Learners from India often comment in the code-mixed language of Hindi-English. Since large language models trained on monolingual text fail to perform satisfactorily on code-mixed data due to the inherent complexity of mixing two languages, there is always an increasing demand for code-mixed datasets to train language models for sentiment analysis. There are few resources available on code-mixed data, but to the best of our knowledge, no public dataset is available on code-mixed data, which particularly focuses on Indian students. This paper introduces SentiGrad, a new Hindi-English code-mixed dataset collected from comments of Indian learners on graduate-level courses on some popular educational channels on YouTube. The comments are labeled into three classes - positive, negative, and neutral, by five independent annotators following the Gold standard guidelines. A preliminary report on the sentiment analysis task on the dataset using the baseline Machine Learning and Deep Learning models along with the state of the art NLP techniques is presented. In addition, we have identified the open challenges which require more sophisticated computational techniques, for the interest of future researchers.