SFANet: Visible-Infrared Person Re-Identification With Shape-Guided Feature Aggregation

Retrieving persons for visible-infrared person re-identification (VI-ReID) task is challenging, due to the huge modality discrepancy induced by the unique imaging principle of RGB and infrared sensors. Existing approaches rely on seeking modality-shared information within unified visual feature space, ignoring the identity-related clues brought by shape information. To facilitate the increasing of interclass distance and the decreasing of intraclass distance so that it can achieve robust retrieval performance, this article proposes a novel shape-guided feature aggregation network (SFANet) to aggregate parsing masks with global-local features. Specifically, we first extract modality-shared and parsing mask features using a parameter-shared backbone and a mask encoder. Then, a part generator (PG) is proposed to refine the implicit information within the parsing masks by separating them into several local pattern maps, and then, a shape-guided feature aggregation module (SFAM) is designed to aggregate the local pattern maps with the group part features through a matching and distributing strategies. Finally, we introduce a part alienation loss (PAL) that provides a diversity of feature space for local pattern maps and group part features, a modality consistency mining loss (MCML) is further introduced for reducing the impact of modality-specific features and maximizing the utilization of modality-invariant features. Our SFANet achieves significant improvements over the state-of-the-art (SOTA) methods on two mainstream datasets.