Hybrid semantic optimization with hierarchical spatial-wavelet synergy for zero-shot anomaly detection

Abstract Zero-shot anomaly detection offers a promising solution for privacy-constrained scenarios. However, the preference of the pre-trained visual encoder for global semantic consistency, together with globally aware prompts, tends to favor object-centric holistic representations, potentially making high- and low-frequency cues that have limited influence on object-level semantics but are important for anomaly detection less adequately represented. In this paper, we propose H2HCLIP, which integrates hybrid semantic optimization with hierarchical spatial-wavelet synergy to enhance frequency-domain visual modeling while introducing semantic guidance. For visual enhancement, we develop the frequency-aware refinement attention at intermediate encoder layers, refining deep-frequency patterns via Fourier transformation, while applying non-saliency completion and threshold-based enhancement to accentuate key representations. At the output stage, spatial-wavelet collaborative inference is conducted, where multiple wavelet transformations intricately merge local high-frequency subtleties with global low-frequency structures. To enrich semantic representations, we introduce image-level, sentiment-level, and region-level prompts to refine normal-abnormal semantic boundaries. We leverage a feature-guided hybrid fusion mechanism and an intra-inter contrastive loss to facilitate the interaction of semantic information from local to global, promoting the joint optimization of classification and segmentation. Extensive experiments on 11 datasets demonstrate that H2HCLIP achieves competitive performance compared with many existing methods, with improvements observed across several evaluation metrics.