Metamorphic Relations-Based Evaluation of Entity Alignment Algorithms in Knowledge Graphs

Entity alignment (EA) algorithms have been developed to identify and use links between entities in knowledge graphs (KGs), which are essential for data integration and information sharing. Given their significance, it is important to thoroughly evaluate EA algorithms to measure their performance on KGs. However, testing methods are still in the early stages, and few studies assess EA algorithms. To address this issue, this article uses the metamorphic testing methodology to automate the testing of EA algorithms. We first design eight metamorphic relations (MRs), grouped into four main groups in the context of EA, which EA algorithms are expected to satisfy. We argue that these eight relations define the necessary or expected properties of EA algorithms. To assess the performance of our proposed MRs, we tested five popular EA algorithms on three widely used datasets using a comprehensive set of metrics. The results show the following: (a) The four MRs influence EA algorithm outputs, with inference-related MR being the most challenging due to advanced reasoning needed. (b) Varying MRs affect EA effectiveness; simpler algorithms like MTransE and SEA perform better on easier datasets when evaluated with MRs. EA algorithms have violated metamorphic relations in some cases, indicating that they still lack certain necessary properties that require further improvement.