RIASA: Enhancing Reasoning Industrial Anomaly Segmentation via Large Vision-Language Models

Industrial anomaly detection (IAD) is crucial for maintaining product quality in manufacturing, but it faces challenges due to limited data and diverse defect semantics. Zero-shot IAD has seen rapid development, while current methods mainly rely on pixel-level anomaly scores, lacking thorough anomaly region definition. The explanatory details related to domain knowledge, such as the anomaly categories, visual semantics, and potential causes and consequences, are often overlooked. Recently, Large Vision-Language Models (LVLMs) have exhibited remarkable perception and generalization abilities across various visual tasks. In this paper, we introduce RIASA, a Reasoning Industrial Anomaly Segmentation Assistant, which integrates reasoning segmentation into IAD via LVLMs. Building on the foundation of reasoning segmentation framework LISA, RIASA has made domain alignment enhancements through instruction-supervised fine-tuning on: (1) semantic segmentation and (2) visual question answering regarding anomaly details. We construct a multi-modal question-answering instruction dataset using GPT-4V, containing 1,890 high-quality question-answer pairs. Extensive experiments on several public 2D and 3D IAD benchmarks demonstrate that RIASA achieves superior performance in zero-shot anomaly segmentation and generalization. Additionally, RIASA’s multi-modal interaction capability provides professional visual descriptions within the IAD domain.