Teacher-Guided Pseudo Supervision and Cross-Modal Alignment for Audio-Visual Video Parsing
Weakly-supervised audio-visual video parsing (AVVP) seeks to detect audible, visible, and audio-visual events without temporal annotations. Previous work has emphasized refining global predictions through contrastive or collaborative learning, but neglected stable segment-level supervision and class-aware cross-modal alignment. To address this, we propose two strategies: (1) an exponential moving average (EMA)-guided pseudo supervision framework that generates reliable segment-level masks via adaptive thresholds or top-k selection, offering stable temporal guidance beyond video-level labels; and (2) a class-aware cross-modal agreement (CMA) loss that aligns audio and visual embeddings at reliable segment-class pairs, ensuring consistency across modalities while preserving temporal structure. Evaluations on LLP and UnAV-100 datasets shows that our method achieves state-of-the-art (SOTA) performance across multiple metrics.
Authors
- Qingyu Luo (ORCID: https://orcid.org/0000-0002-1980-4725)
- Zhenbo Li (ORCID: https://orcid.org/0000-0003-2914-1192)
- Liting Gao (ORCID: https://orcid.org/0009-0004-6659-382X)
- Ruohao Guo (ORCID: https://orcid.org/0000-0002-1091-272X)
- Yang Xiang (ORCID: https://orcid.org/0000-0001-5252-0831)
- Wenwu Wang (ORCID: https://orcid.org/0000-0002-8393-5703)
- Yaru Chen (ORCID: https://orcid.org/0000-0002-1917-8752)
Institutions
- Peking University (CN)
- University of Surrey (GB)
- Peer Intelligence Technology (China) (CN)
- China Agricultural University (CN)
Publication Details
- Published
- 2026-04-21
- DOI
- https://doi.org/10.1109/icassp55912.2026.11463157
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- China Scholarship Council