Distillation and prompt in Chinese CLIP model

Contrastive Language Image Pre (CLIP) is a promising language supervised visual pre training framework. In recent years, there have been many studies on strengthening CLIPs. However, there is a lack of reinforcement methods related to pre training models for contrastive language images in Chinese. This article aims to propose an efficient and accurate method for image text mutual inspection model that combines the Chinese CLIP model, prompt technology, and knowledge distillation technology. This method first implements Chinese CLIP with prompt support, and then distills and refines it using a small Chinese CLIP model supervised by a large teacher CLIP model. Through the comparison of recall rates, we find that the proposed model improved R @ 1 by 15%, R @ 5 by 13.3%, and R @ 10 by 10.2% compared to the baseline Chinese CLIP model. Through experiments, it has been found that interactive contrastive learning between teachers and student coders is effective in improving performance. We can attribute the success of Chinese CLIP-P-KD to maximizing feature similarity between teachers and students.