Occlusion-Aware Joint Matching: A Study on Multimodal Face-Pedestrian Association for Multi-Object Tracking

Complex Multi-Object Tracking suffers from identity switches and fragmentation due to appearance similarity and occlusion. Current methods lack robust cross-category (e.g., face-pedestrian) matching. To address this, we propose an improved FairMOT model for multi-class face-pedestrian tracking. Our approach features a novel joint spatiotemporal-appearance metric integrating historical centroid distance, IoA/IoU constraints, cross-target feature validation, and occlusion-aware weight adaptation. Matching uses Hungarian-algorithm bipartite graph optimization. Evaluations on Chokepoint_Face_Pedestrian_Track show significant superiority over traditional methods (IoU, IoA, FP), with about 4–43 pp F1 AUC gains in complex scenarios, confirming enhanced robustness and accuracy.