Multi-object tracking review: retrospective and emerging trend

Multi-object tracking (MOT) is a critical task involving detecting and continuously tracking multiple objects within a video sequence. It is widely used in various fields, such as autonomous driving and intelligent security. In recent years, deep learning architectures have effectively promoted the development of MOT. However, this task poses significant challenges regarding accuracy due to occlusion/truncation, light variation, camera movement. Researchers have proposed many methods to address these issues to reduce trajectory fragmentation, identity switches, and missing targets. To better understand these advancements, it is essential to categorize the approaches based on their methodologies. This article reviewed the recent development of MOT, divided into Tracking by Detection (TBD) and End-to-End (E2E). By introducing and comparing the two types of tracking algorithms, readers can quickly understand the current development status of MOT. Meanwhile, this review summarizes the links to open-source code of excellent algorithms and common benchmark datasets in the appendix. And provide a unified MOT toolkit that includes evaluation and visualization at https://github.com/guanzhiyu817/MOT-tools . In addition, this review discusses the future directions of MOT, specifically cross-modal reasoning.