Vulnerabilities and Defenses in Audio-Visual Attacks: A Survey From Audio to Multimodal Models

With the rapid development of multimodal large language models (MLLMs) and the cost of large amounts of data and resources, researchers are more inclined to use public datasets and fine-tune open-source MLLMs to achieve excellent performance in audiovisual related tasks. While this trend accelerates progress, it also introduces security risks. Empirical studies demonstrate that the latest MLLMs can be manipulated to generate harmful content through crafted inputs, including adversarial perturbations and malicious queries, bypassing internal safeguards. To gain a deeper comprehension of the inherent security vulnerabilities associated with audio-visual-based multimodal models, a series of surveys investigates various types of attacks, including adversarial and backdoor attacks. While existing surveys on audio-visual attacks provide a comprehensive overview, they are limited to specific types of attacks, which lack a unified review of various types of attacks. To bridge this gap, this paper presents a comprehensive and systematic review of audio-visual attacks, which include adversarial, backdoor, and jailbreak attacks. Furthermore, we also review various types of attacks in the latest audio-visual-based MLLMs, a dimension notably absent in existing surveys. Drawing from extensive literature, this paper delineates both challenges and emergent trends for future research on audio-visual attacks and defenses.