Hybrid Token Optimization for Text-to-Image Person Re-identification

Abstract Text-to-image person re-identification (TIReID) enables natural-language search over pedestrian image collections. Although dual-grained methods combine global and local matching, recent dual-grained pipelines use encoder-produced global tokens for coarse matching and attention-ranked local tokens for fine-grained matching. This interface leaves global representations without complementary priors, local evidence dependent on a single ranking cue, and image--text sequence asymmetry insufficiently addressed. To address these limitations, we propose Hybrid Token Optimization (HTO), a parameter-efficient, role-aware framework at the encoder--matcher interface. HTO comprises two coordinated components. Dictionary Prior Enhancement uses modality-specific dictionaries, cross-attention, and conservative residual blending to refine global-token representations. Spatial-Energy Heterogeneous Token Selection combines attention-based saliency, a feature-norm cue, and an adaptive minimum-coverage constraint to broaden local-token coverage in both modalities. Both components leave the established matching heads unchanged and operate during both training and inference, while the auxiliary losses are used only for training. Under a fixed-seed protocol, HTO yields Rank-1 point-estimate gains of 0.45, 0.40, and 2.25 percentage points over the reproduced baseline on CUHK-PEDES, ICFG-PEDES, and RSTPReid, respectively. It increases the parameter count by less than 0.4% and is also compatible with Test-Time Human-Centered Interaction. These results suggest that role-aware token-interface refinement can improve fine-grained cross-modal retrieval without redesigning the matcher.