ReSORT: LLM-Guided Policy Selection for New-ID Assignment Under Occlusion in Multi-Object Tracking

Objects that reappear after occlusion can receive new identities when data association fails, fragmenting trajectories. We present ReSORT, an identity recovery framework that evaluates competing links using motion history, numerical association scores, and selective language-model evidence. Camera-compensated trajectory states are propagated through five motion hypotheses to construct a persistent candidate graph. A numerical ranker scores candidate links, and a frozen Qwen3.5-4B model reviews selected ambiguous cases using image crops and scene context. Candidate-order consistency checks filter the model responses before their relative evidence enters a global one-to-one assignment. Inference does not access ground truth; the numerical ranker is trained with supervision, and the evaluated reassociation uses subsequent observations offline. On seven MOT17 development videos, the complete pipeline produces 243 identity switches, compared with 427 for native ByteTrack and 299 for the historical automatic ReSORT reference, with IDF1 of 85.96% and HOTA of 78.06%. The historical comparison includes changes beyond the final language-model component, whose contribution is assessed separately through matched ablations. Experiments across seven tracker families on MOT17 and MOT20 show tracker-dependent benefits and sensitivity to evidence weighting. The results support selective language-model verification within a constrained association system while identifying limitations in response consistency, computational cost, and development-set evaluation.

Authors

Institutions

Publication Details

Journal
Applied Sciences
Published
2026-11-18
DOI
https://doi.org/10.3390/app162211445
Primary Topic
Video Surveillance and Tracking Methods
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

ReSORT: LLM-Guided Policy Selection for New-ID Assignment Under Occlusion in Multi-Object Tracking

Subin Oh, Daesung Lee, Chigon Hwang, yujin roh et al.
2 links
Applied Sciences
Video Surveillance and Tracking Methods
article

ReSORT: LLM-Guided Policy Selection for New-ID Assignment Under Occlusion in Multi-Object Tracking

Subin Oh, Daesung Lee, Chigon Hwang, yujin roh, Inho Jake Park
article en

Abstract

Objects that reappear after occlusion can receive new identities when data association fails, fragmenting trajectories. We present ReSORT, an identity recovery framework that evaluates competing links using motion history, numerical association scores, and selective language-model evidence. Camera-compensated trajectory states are propagated through five motion hypotheses to construct a persistent candidate graph. A numerical ranker scores candidate links, and a frozen Qwen3.5-4B model reviews selected ambiguous cases using image crops and scene context. Candidate-order consistency checks filter the model responses before their relative evidence enters a global one-to-one assignment. Inference does not access ground truth; the numerical ranker is trained with supervision, and the evaluated reassociation uses subsequent observations offline. On seven MOT17 development videos, the complete pipeline produces 243 identity switches, compared with 427 for native ByteTrack and 299 for the historical automatic ReSORT reference, with IDF1 of 85.96% and HOTA of 78.06%. The historical comparison includes changes beyond the final language-model component, whose contribution is assessed separately through matched ablations. Experiments across seven tracker families on MOT17 and MOT20 show tracker-dependent benefits and sensitivity to evidence weighting. The results support selective language-model verification within a constrained association system while identifying limitations in response consistency, computational cost, and development-set evaluation.

Applied SciencesVol. 16(22)
Tech University of Korea (KR), Korea Maritime and Ocean University (KR)
Video Surveillance and Tracking Methods
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon