Semantic-Preserving Dense Localization via DINO-Guided Refinement of CLIP Anomaly Maps

CLIP-based zero-shot anomaly detection has recently shown strong generalization by aligning visual features with textual normal/abnormal concepts. However, CLIP has essentially been trained for image-level text alignment, making it difficult to ensure spatially consistent localization of local defective regions using only similarities between text anchors (meaning “normal/abnormal”) and patch tokens. Therefore, dense anomaly localization remains challenging, and the anomaly maps produced by CLIP-based detectors are often unstable, fragmented, and spatially noisy, particularly under cross-dataset transfer. This paper investigates how DINOv2 can improve CLIP anomaly maps without disrupting their language-aligned anomaly semantics. Our key observation is that DINO features are more effective as score-free structural priors than as independent anomaly scorers. Based on this observation, we propose an inference-time DINO-guided refinement framework. Specifically, we construct a DINO patch affinity graph and propagate CLIP anomaly maps over this graph. This propagation suppresses isolated noisy responses while aggregating semantically similar fragmented responses into more coherent anomaly regions, thereby improving structural consistency without modifying the CLIP anomaly scorer. We further introduce a CLIP-gated DINO memory residual that uses normal-memory distance as complementary evidence after graph-based refinement. Extensive experiments on industrial benchmark datasets showed that DINO-guided affinity propagation consistently improves pixel-level localization across multiple CLIP-based detectors. In contrast, generic spatial smoothing reduced map noise but degraded localization performance, indicating that effective refinement requires structure-aware propagation. The CLIP-gated memory residual further improved anomaly localization, demonstrating the benefit of separating CLIP-based anomaly semantics from DINO-based structural refinement.