Sentimental Awareness Influence on Paraphrase Detection in Transformer Models

In recent years, natural language processing has had notable development with BERT emerging as a useful tool for linguistic tasks. This study investigates the effect of sentiment analysis integration in paraphrase detection models, specifically utilizing the transformer-based language models BERT, RoBERTa, DeBERTa, and ELECTRA. The developed models follow a common strategy for text similarity consisting of machine learning with cosine similarity values measured from vectors created by the language models. The models employed from standard machine learning paraphrase detection practices with cosine similarity of vector representation generated from the language models are improved through the incorporation of sentiment information. Resulting from a chain of experiments carried out on multiple datasets and evaluation metrics, we observe the influence of sentiment awareness on the performance of accuracy on simple sentence similarity tasks. This was executed by training machine learning models consisting of both semantic similarity and sentiment similarity values for each pair of sentences. Sentiment analysis incorporated provides a polarity similarity for pairs of sentences. The research demonstrates the integration of sentiment analysis had a positive effect on the tested paraphrase detection models. The research suggests that sentiment, or the emotional tone of text, can assist in constructing well performing paraphrase detection tools. It emphasizes questions on the importance of linguistic characteristics and their relevance for specific tasks, particularly as natural language processing models continuously develop more understanding of linguistic complexities. Additionally, the study highlights the importance of researching what factors contribute to NLP task performance.