DDAT-Net: Dual Dynamic Agent Transformer Network for Audio-Visual Event Localization

Audio-Visual Event Localization (AVE) aims to identify the temporal segments where specific events occur by jointly analyzing audio and visual modalities. This task demands effective integration of multimodal information to achieve accurate localization. However, existing approaches often face challenges in modeling complex cross-modal interactions and dynamic dependencies, leading to limited performance. To address these issues, we propose the Dual Dynamic Agent Transformer Network (DDAT-Net), which introduces a novel Dual Dynamic Agent Transformer mechanism. This mechanism is composed of two key modules: the Dual Agent Transformer module and the Dynamic Contextual Cross Modal Attention module. The Dual Agent Transformer module dynamically learns modality-specific features, while the Dynamic Contextual Cross Modal Attention module captures fine-grained cross-modal dependencies through adaptive attention allocation. By leveraging these dual attention mechanisms, DDAT-Net effectively balances modality-specific learning and cross-modal fusion, enabling precise event localization. Extensive experiments conducted on the AVE benchmark dataset demonstrate that DDAT-Net achieves superior performance compared to existing state-of-the-art methods. Our work underscores the potential of Dual Dynamic Agent Transformer mechanisms in advancing the field of audio-visual event localization.