Text-Scene Retrieval for Driving Scenes in Transportation Cyber-Physical Systems

The widespread application of intelligent connected vehicles poses challenges to the security and reliability of transportation cyber-physical systems. How to retrieve high-quality samples from vast amounts of natural driving data has emerged as a significant research direction in the field of intelligent transportation data closed-loop systems. This paper proposes an Entity and Relation-Aware Cross-Modal Retrieval (ERA-CR) framework, addressing the limitations of traditional video retrieval methods in representing entity relationships and aligning complex semantics in traffic scenarios. In the visual modality, DETR3D and Graph Convolutional Networks (GCN) are utilized to extract object-level features and construct spatial relationship graphs, capturing interactions among dynamic entities. For the textual modality, a T5-based model, combined with GCN, transforms natural language descriptions into structured semantic graphs. A Transformer-based dynamic multi-scale fusion module integrates global and local features, facilitating fine-grained semantic alignment. Additionally, a contrastive learning-based alignment loss ensures robust consistency in the shared embedding space. Experiments on the NuScenes dataset demonstrate the effectiveness of ERA-CR, achieving R@1 of 62.7% and MAP of 75.6%, significantly outperforming state-of-the-art baselines. Ablation studies further validate the contributions of the proposed modules, offering a robust solution for cross-modal retrieval in T-CPS applications.