Content-based image similarity measurement grounded on information retrieved by semantic segmentation algorithms

<ns3:p>The purpose of this article is to present a novel approach for recording information contained in an image in a structured form and performing image similarity assessment with use of these data structures. The solution presented in this document relies on an analysis of results produced by pre-trained semantic segmentation algorithms. These outcomes can be transformed to a set of vectors representing some characteristics of each class of objects detected in the provided image. These data structures can contain meaningful information about algorithm detections, such as the object’s position on the image, the object’s size compared to the overall image size or the object’s dominant colors, etc. Vectors prepared as described previously can be further compared with other image embeddings using many mathematical tools like distance measures. Moreover, the approach described in this article allows the user to define a value of weight tied to each characteristic. This provides the ability to make a subset of features more important than others and have a greater impact on the final value of image similarity.</ns3:p>