Combining SIFT and BRISK Descriptors to Improve Image Matching Accuracy

Local descriptor algorithms are foundational in computer vision applications such as image matching and image retrieval. Some local descriptor algorithms extract features containing similar information from images while others extract complementary information. In this work, we investigate the advantages of combining a binary and a non-binary local descriptor algorithm. We propose and compare three methods to combine SIFT and BRISK descriptor algorithms selected because they produce complementary descriptor vectors. First, we propose to combine SIFT and BRISK descriptors using a weighted summation of their individual descriptor distances with learned weights. Our second method converts SIFT into a binary descriptor and concatenates the binary SIFT vector with the BRISK descriptor. The third method is to scale the binary BRISK descriptor vector and concatenate it with the SIFT descriptor. Parameters for combining the descriptors are learned based on the HPatches data set for each of the three methods. Our proposed methods increase the mean Average Precision in the range of 3% to 15.8% over the original BRISK and in the range of 5.9% to 21.8% over the original SIFT algorithm in various evaluation conditions.