Self-Consistent Semantic Feature Extraction of Image Object Based on Contrastive Learning

Contrastive learning has become an influential paradigm in the field of Computer Vision, and great progress has been made in feature extraction by exploiting the comparison of positive and negative samples. However, how to make the most of the features of different parts of the same object to infer more accurate similarity scores is still underexplored. One way to alleviate the problem is to use the framework of contrastive learning to study how to improve the feature similarity of different parts of the same object in the latent space and find out more features with high similarity called “self-consistent semantic features”. In this paper, we propose a neural network model SSFE for self-consistent semantic feature extraction, and a training method by retraining the hard dataset is also included. Specifically, the representation vectors are learned to describe local features better, and then a neural network module is introduced to infer similarity by mapping the stacked feature vectors to the sample space. Furthermore, another module is developed to help learn the representation vectors by classifying them into different classes. Our experiments show that our model with the training method can extract better self-consistent semantic features in PASCAL VOC 2007 and PASCAL VOC 2012.