A study on Chinese language Cross-Modal pedestrian image information retrieval

This research aims to achieve cross-modal Chinese pedestrian information retrieval (CMCPIR). Most PIR tasks focus on English due to data availability, as to the best of our knowledge, there is no available Open access Chinese dataset or PIR. Therefore, we built a Chinese dataset (CUHK-PEDES-CHINESE) using English CUHK-PEDES dataset. We constructed a baseline CPIR model (HCMacBRes50) by modifying a top performing English PIR model. We have experimented and modified a set of deep learning models based on image model and Chinese language text model to construct a novel RoberResX4D CMCPIR model. Furthermore, we designed a novel MSWCMPME loss function to enhance the performance of our CMCPIR model by optimizing the fusion strategy for the outputs of image-text model. The RoberResX4D model has the same PIR performance and is more robust than the English model of SRCF. The RoberResX4D model effectively combines information from different modalities.