Arabic Sentence Similarity Based on Attention Constituency Vector Tree

Sentence similarity assesses how two sentences are similar to each other in terms of their lexical and semantic similarities. However, the Arabic language presents a challenge due to the richness of its features and analysis ambiguities. In this context, we describe syntactic–semantic similarity based on chunks analysis for Arabic paraphrases using partial tree kernels and attention constituency vector tree (ACVT) kernel. For each sentence, a parsing tree is created to extract the main chunks. Then, chunks embeddings are generated using Aravec pre-trained embeddings. Finally, the similarity is measured using the updated ACVT kernel formula to consider the semantic information and the weights provided with the main chunks in the constituent tree instead of being with the leaves of the tree (i.e. words). The proposed method is evaluated using an Arabic paraphrasing benchmark and compared with [Formula: see text]-gram chunks similarity. Several experiments are conducted to exploit the effect of the decay factors in the tree kernel function. The proposed method achieves better correlation than the [Formula: see text]-gram chunk method. Furthermore, the method can be applied to paraphrase detection and achieves a recall of 0.70 and a precision of 0.757, with a similarity threshold of 0.5, while the results of paraphrase detection with a threshold of 0.3 show a better recall of 0.94 and a precision of 0.751.