Zero sample image anomaly detection guided by multi-modal prompts

Large visual language models such as CLIP show strong zero sample recognition capability in anomaly detection tasks. It learns object-independent prompts through the guidance of prompt engineering, fine-tunes CLIP to anomaly detection tasks, and shows good generalization ability. However, prompt engineering usually needs to design hundreds of prompts, which increases the difficulty of designing prompts, and can not be used well for multi-class anomaly detection. To solve these problems, this paper proposes a zero sample anomaly detection method guided by multi-modal prompt learning. By introducing multimodal prompts for text branches and visual branches, the consistency of text and visual branch representations is improved, ensuring that the two modes learn cooperatively from each other. In addition, a uniform domain-aware contrast state template is designed for fine-grained visual language matching. For imagelevel and pixel-level anomaly detection, this method achieves better performance than non-prompt learning in zero-sample anomaly detection.