Comparison of Similarity Functions for n-gram Software Birthmarks

Various software birthmarks have been proposed to detect software plagiarism or unauthorized reuse of code. If two software products have very similar birthmarks, the existence of reused code between them is strongly suspected. However, the accuracy of detecting reused code depends on the similarity function for comparing birthmarks and the threshold for distinguishing reused and non-reused codes. Moreover, since software products usually consist of many program modules such as source files and class files, an appropriate method to compare the set of modules between two software products is needed. In this paper we propose a method for comparing birthmarks between two software products containing many modules, as well as a method for determining the threshold for distinguishing reuse. Afterwards, focusing on the n-gram birthmarks, which is one of the most easy-to-measure birthmarks, we empirically compare 4 similarity functions: cosine similarity, Dice index, Jaccard coefficient and Simpson index. In the experiment, we employ multiple versions of 15 open-source software (OSS) projects of 5 categories, making sure that they contain project pairs that contain reused code and that do not contain reused code. The experimental result shows that, for both 3-gram and 6-gram birthmarks, Jaccard coefficient showed the best performance in terms of F1-score to identify reused code.