Leveraging Large Language Model for Robust Industrial Image Anomaly Detection

Currently, large visual language models (LVLMs), such as MiniGPT-4 and LLaVA, have demonstrated the ability to understand images and have achieved outstanding performance across various visual tasks. Despite the impressive performance of large visual language models (LVLMs) in image understanding, they still exhibit limitations in specific domains, such as industrial anomaly detection (lAD). Particularly when dealing with tasks that require specialized knowledge, these models often struggle to capture subtle details in images, resulting in unsatisfactory performance in the lAD field. Moreover, existing lAD solutions typically provide only a scoring and approximate location of anomalous objects, relying on manually set thresholds to determine whether an object is anomalous. This approach has limited generalization capabilities and often fails to meet expectations in practical applications. In this study, we explore the potential of integrating LVLMs with traditional lAD methods to address these issues. Specifically, we selected LLa V A as the foundational large visual language model (L VLM) and designed a visual decoder with an intra-class adaptive scoring mechanism to address L VLM's limitations in capturing fine details. Our research successfully eliminates the need for manual threshold settings, achieving a fully automated anomaly detection and localization process. Through extensive experiments on the MVTec-AD[l] and VisA[2] datasets, our method demonstrated outstanding detection performance, thereby validating its effectiveness.