Detecting Empathy in Roman Urdu Using Transfer Learning

Empathy means to understand the other person’s feelings and emotions to become familiar with their situation. This work aims to create a Roman Urdu empathy dataset containing two types of sentences, either “Depressed” or “Not-Depressed,” and execute different machine learning, deep learning, and pre-trained models on the dataset for experimentation. The dataset has been created by extorting depressed Roman Urdu and motivational sentences from web pages and social media websites such as Facebook, Twitter, and YouTube. Multiple evaluation results led to the observation that the extraction of attributes from the union of word- and character-level features improves the accuracy of machine learning algorithms. The extraction of features by the unification of character-level unigram, bigram, trigram, and 5-gram feature extraction methods demonstrated improved results. In contrast, character-level unigrams and word-level trigrams have retrieved fewer significant parameters from sentences. Naive Bayes and Random Forest have exhibited the lowest accuracies on word-level, character-level, and the union of word- and character-level features as well. However, Logistic Regression, Linear Support Vector Classifier, and Support Vector Machine have obtained better accuracy in all three feature extraction methods. The perceptron-based algorithm has illustrated improved results than most of the machine learning methods. Almost all deep learning algorithms produce results that are a step ahead of those of machine learning and perceptron-based models. Analysis of deep learning experiments shows that FastText word embeddings support the models in achieving high accuracy. Recurrent Neural Network, and a combination of Convolutional Neural Network and Gated Recurrent Unit, performed well overall on all word and sentence embeddings. However, Recurrent Convolutional Neural Network had the lowest final results across all three types of embeddings. Observing the outputs of pre-trained models revealed that they produce more precise findings than many machine learning and deep learning models. In comparison to all pre-trained models, Universal Language Model Fine-Tuning has outperformed the rest. Hence, the results show that pre-trained language models have produced outstanding results on the Roman Urdu empathy dataset.