Digital twin-assisted graph matching multi-task object detection method in complex traffic scenarios

Addressing the challenges of time-consuming and labor-intensive traffic data collection and annotation, along with the limitations of current deep learning models in practical applications, this paper proposes a cross-domain object detection transfer method based on digital twins. A digital twin traffic scenario is constructed using a simulation platform, generating a virtual traffic dataset. To address distributional discrepancies between virtual and real datasets, a multi-task object detection algorithm based on graph matching is introduced. The algorithm employs a graph matching module to align the feature distributions of the source and target domains, followed by a multi-task network for object detection. An attention mechanism is then applied for instance segmentation, with the two tasks exhibiting different noise patterns that mutually enhance the robustness of the learned representations. Additionally, a multi-level discriminator is designed, leveraging both low- and high-level features for adversarial training, thus enabling tasks to share useful information, which improves the performance of the proposed method in object detection tasks. Through comprehensive comparative experiments with various state-of-the-art methods, the practical value of the generated virtual dataset has been fully demonstrated. Furthermore, the effectiveness of the proposed graph-matching-based transfer method has been validated. These findings highlight the dataset's capacity to enhance task performance and underscore the robustness and adaptability of the proposed approach in diverse scenarios.