Unify and Boost Token for Multi-Modal Person Re-Identification

Multi-modal person re-identification aims to match images of the same pedestrian captured by different cameras at different locations, utilizing multiple modalities of the pedestrian's images. The advantage of multi-modal images lies in their ability to provide complementary information between modalities, thereby alleviating the problem of insufficient representation caused by factors such as lighting in single-modal images. Recent works have shown that fusing multi-modal features can improve the performance of models in person re-identification tasks. However, most of these works do not narrow the distribution gap between modalities before fusing the features, and they do not fully exploit the information provided by local features during the fusion stage. Therefore, we propose a multi-modal person re-identification framework that utilizes trainable prototypes to integrate and align tokens extracted from various modalities by Vision Transformers before employing a multi-head cross-modal attention mechanism for cross-modal interaction. The framework consists of three parts: Baseline Feature Extraction based on Vision Transformers, Local Feature Alignment Learning Module, and Attention-guided Cross-domain Fusion Module. Firstly, Baseline Feature Extraction based on Vision Transformers comprehensively extracts various features from each modality to avoid information loss. Secondly, Local Feature Alignment Learning Module integrates and aligns the tokens extracted from each modality, reducing redundancy in the information. Finally, the aligned tokens are further enhanced in their feature representation and robustness through Attention-guided Cross-domain Fusion Module. Experimental results on the RGBNT201 dataset demonstrate the effectiveness of our proposed approach, achieving promising performance compared to existing methods in recent works.