CPES: A Comprehensive Method for Automatic Evaluation of Paraphrased Sentences

Paraphrasing is the process of transforming a given text into another text using alternative lexical or syntactic forms while preserving its original meaning. Paraphrasing significantly affects several Natural Language Processing (NLP) applications, such as machine translation (MT) and data augmentation. Paraphrasing suffers from a specifically designed metric, and most research adopts metrics developed for other NLP purposes. Paraphrase evaluation remains challenging due to the limitations of surface-level similarity metrics such as BLEU and ROUGE. Therefore, this research aims to develop a new metric for paraphrase generation, the Comprehensive Paraphrasing Evaluation Score (CPES). Furthermore, the CPES requires lexical language resources; thus, the research uses an Arabic corpus and produces a new Arabic lexical dictionary (Rabih dictionary). The CPES considers major paraphrasing criteria, including sentence structure, changes in word forms, synonym substitution, and paraphrased-sentence lexical diversity (LD). Each CPES supports interpretability by enabling decomposition into the criterion that drives the final result. The research finds that (1) the CPES effectively measures the modification ratio between original and paraphrased sentences, and (2) the text category impacts the CPESs.