FASDP: Frequency-Aware Scale-Diverse Purification against Pixel-Space Visual Jailbreaks

Abstract Vision-language models (VLMs) have achieved remarkable success across a wide range of applications.However, as their deployment becomes increasingly widespread, recent studies have revealed that VLMs are vulnerable to visual jailbreak attacks, where adversarially manipulated images induce safety-aligned models to produce harmful responses, posing serious threats to real-world security.Input purification defenses must balance perturbation suppression against content preservation; a fixed transformation strength may be unsuitable for inputs and attacks with different characteristics.We propose FASDP, a Frequency-Aware Scale-Diverse Purification framework that adaptively allocates purification strength per image.It fuses multi-scale candidates from a single frozen denoiser using low-frequency consistency modulated by a self-referenced frequency cue.The framework requires no task-specific training or access to the target model.We evaluate FASDP on LLaVA-1.5-7B, InstructBLIP-7B, and Qwen2.5-VL-7B under three representative pixel-space attacks.FASDP attains strong protection under these attacks while keeping clean-image performance on downstream tasks close to that of the undefended models.