Temporal Motion and Spatial Enhanced Appearance with Transformer for video-based person ReID

For video-based person Re-Identification (Re-ID), how to efficiently extract temporal motion features and spatial appearance features from video sequences is a key issue. Conventional approaches focus on modelling the entire video spatio-temporal features, ignoring the inherent differences between temporal motion features (e.g., gait) that change over time and spatial appearance features (e.g., clothing) that are stable over time in terms of attributes. Because of their different sensitivities in real-world scenarios, conventional approaches often lose critical fine-grained features. To address these issues, we propose a T emporal M otion and spatial E nhanced A ppearance with T ransformer-based (T 2 MEA) framework for modelling spatial–temporal video discriminative representations. Specifically, (1) Dual-Branch Architecture: The content branch emphasises extracting the overall structure of the video using the spatial–temporal aggregation (STA) module from a global view, whereas the fovea branch focuses on gaining local fine-grained spatio-temporal features. (2) Zero-Parameter Design: the [CLS] Token Channel Shift Interaction (TCSI) module captures the dynamic features and static features between adjacent frames without additional parameters; the Spatial Patches Shift Enhancing (SPSE) module is introduced to enhance appearance features within frame to address occlusion and illumination changes without additional parameters. (3) Spatial–Temporal Interaction: The Cross-Attention Aggregation (CAA) module is proposed to interact between temporal and spatial features and further enrich the spatial–temporal feature representation for video sequences. Extensive experiments on three public Re-ID benchmarks (MARS, iLIDS-VID, and PRID-2011) demonstrate that the proposed framework outperforms several state-of-the-art methods. • A novel framework combines temporal motion and spatial appearance features for video-based person re-identification. • It introduces zero-parameter design to capture dynamic/static features and enhance appearance features without additional parameters. • It utilises Cross-Attention Aggregation method to fuse temporal and spatial features, obtaining comprehensive spatio-temporal feature representation.