What a Passing Reproduction Check Does Not Certify: A Gated PatchCore Reference on MVTec AD 2

A reproduction check, matching a method's published score on its original benchmark, is the usual safeguard before a pipeline is trusted on a new one. We show that it can pass and still miss what decides the verdict on the new dataset. Our full-shot PatchCore pipeline built its memory bank from antialiased images and scored test images resized without antialiasing. It cleared its MVTec AD check 0.02 points inside the tolerance (98.02 ± 0.07 I-AUROC against a published 99.0, ±1.0), under a seed-mean rule written after a first single-seed run had missed; the corrected pipeline scores 98.22. On the seed mean, only a tolerance from 0.78 to just under 0.98 separates the two; a per-seed rule would have rejected the first, on one seed of three, by 0.06 points. On MVTec AD 2 it left Sheet Metal at chance (0.510). Re-running both configurations on one runtime with the same seeds isolates the resize as the cause on these images. Correcting it lifts Sheet Metal to 0.797, a paired change (+0.287) that stays above zero under resampling of test scenes. It lifts Fabric from 0.562 to 0.723, a change whose interval includes zero. We report the corrected pipeline on all eight categories of the public split; by a detection-only criterion fixed after the numbers existed, it is a usable reference on five. The planned zero-shot comparison stopped at its gate: WinCLIP failed its check, so we report no MVTec AD 2 number for it. Preprint, version 1, November 2026; not peer-reviewed. Licensed under CC BY 4.0, except Figs. 2 and 3, which contain images from MVTec AD 2 (© 2025 MVTec Software GmbH) under CC BY-NC-SA 4.0 and are excluded from it.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-03
DOI
https://doi.org/10.5281/zenodo.23932258
Primary Topic
Industrial Vision Systems and Defect Detection
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

What a Passing Reproduction Check Does Not Certify: A Gated PatchCore Reference on MVTec AD 2

Sergio Alejandro Duarte Pacheco
Zenodo (CERN European Organization for Nuclear Research)
Industrial Vision Systems and Defect Detection
preprint

What a Passing Reproduction Check Does Not Certify: A Gated PatchCore Reference on MVTec AD 2

Sergio Alejandro Duarte Pacheco
preprint en

Abstract

A reproduction check, matching a method's published score on its original benchmark, is the usual safeguard before a pipeline is trusted on a new one. We show that it can pass and still miss what decides the verdict on the new dataset. Our full-shot PatchCore pipeline built its memory bank from antialiased images and scored test images resized without antialiasing. It cleared its MVTec AD check 0.02 points inside the tolerance (98.02 ± 0.07 I-AUROC against a published 99.0, ±1.0), under a seed-mean rule written after a first single-seed run had missed; the corrected pipeline scores 98.22. On the seed mean, only a tolerance from 0.78 to just under 0.98 separates the two; a per-seed rule would have rejected the first, on one seed of three, by 0.06 points. On MVTec AD 2 it left Sheet Metal at chance (0.510). Re-running both configurations on one runtime with the same seeds isolates the resize as the cause on these images. Correcting it lifts Sheet Metal to 0.797, a paired change (+0.287) that stays above zero under resampling of test scenes. It lifts Fabric from 0.562 to 0.723, a change whose interval includes zero. We report the corrected pipeline on all eight categories of the public split; by a detection-only criterion fixed after the numbers existed, it is a usable reference on five. The planned zero-shot comparison stopped at its gate: WinCLIP failed its check, so we report no MVTec AD 2 number for it. Preprint, version 1, November 2026; not peer-reviewed. Licensed under CC BY 4.0, except Figs. 2 and 3, which contain images from MVTec AD 2 (© 2025 MVTec Software GmbH) under CC BY-NC-SA 4.0 and are excluded from it.

Zenodo (CERN European Organization for Nuclear Research)
Industrial Vision Systems and Defect Detection
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
What a Passing Reproduction Check Does Not Certify: A Gated PatchCore Reference on MVTec AD 2 — Sergio Alejandro Duarte Pacheco · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS