Advancing approaches to multi-target multi-camera tracking: Graph-based features, language integration, privacy preservation, and large language agent frameworks

Multi-target multi-camera tracking (MTMCT) is a cornerstone of intelligent transportation systems (ITS), enabling comprehensive monitoring and analysis of vehicle movements across distributed camera networks. Although significant progress has been made, fundamental challenges persist in data association, real-time processing, preservation of privacy, and natural-language user interaction. First, we propose a graph-based model that leverages similarity algorithms to enhance cross-camera object association. Our framework achieves state-of-the-art performance with an IDF1 score of 0.8166 on the CityFlow dataset for offline tracking while maintaining real-time capability at 14 FPS for online scenarios. Next, we present LaMMOn, an end-to-end framework integrating language models with graph neural networks. LaMMOn addresses data scarcity by generating synthetic embeddings, demonstrating competitive results in multiple datasets, including CityFlow (HOTA 76.46%) and TrackCUIP (HOTA 80.94%). To enable privacy-preserving tracking in large-scale deployments, we develop FLaMMOn, a federated learning framework incorporating federated elastic weight consolidation (FedCurv) and federated representation learning (FedRep). FLaMMOn outperforms centralized approaches with an IDF1 score of 76.04% while ensuring robust privacy guarantees. Finally, we introduce MACA, a large language multi-agent model that enables natural language for user queries for MTMCT (e.g., “Track black sedans moving from Camera 1 to Cameras 3 and 6 between 2 PM and 5 PM”). MACA achieves a HOTA score of 66.23% on our newly introduced Refer-CityFlow dataset. The comprehensive solutions presented in this dissertation enhance MTMCT systems through improved accuracy, scalability, privacy preservation, and user interaction capabilities. The proposed frameworks establish new benchmarks in performance while addressing critical real-world deployment challenges, paving the way for more effective and secure intelligent transportation systems.