What a Passing Reproduction Check Does Not Certify: A Gated PatchCore Reference on MVTec AD 2
A reproduction check, matching a method's published score on its original benchmark, is the usual safeguard before a pipeline is trusted on a new one. We show that it can pass and still miss what decides the verdict on the new dataset. Our full-shot PatchCore pipeline built its memory bank from antialiased images and scored test images resized without antialiasing. It cleared its MVTec AD check 0.02 points inside the tolerance (98.02 ± 0.07 I-AUROC against a published 99.0, ±1.0), under a seed-mean rule written after a first single-seed run had missed; the corrected pipeline scores 98.22. On the seed mean, only a tolerance from 0.78 to just under 0.98 separates the two; a per-seed rule would have rejected the first, on one seed of three, by 0.06 points. On MVTec AD 2 it left Sheet Metal at chance (0.510). Re-running both configurations on one runtime with the same seeds isolates the resize as the cause on these images. Correcting it lifts Sheet Metal to 0.797, a paired change (+0.287) that stays above zero under resampling of test scenes. It lifts Fabric from 0.562 to 0.723, a change whose interval includes zero. We report the corrected pipeline on all eight categories of the public split; by a detection-only criterion fixed after the numbers existed, it is a usable reference on five. The planned zero-shot comparison stopped at its gate: WinCLIP failed its check, so we report no MVTec AD 2 number for it. Preprint, version 1, November 2026; not peer-reviewed. Licensed under CC BY 4.0, except Figs. 2 and 3, which contain images from MVTec AD 2 (© 2025 MVTec Software GmbH) under CC BY-NC-SA 4.0 and are excluded from it.
Authors
- Sergio Alejandro Duarte Pacheco (ORCID: https://orcid.org/0009-0009-7079-0160)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-03
- DOI
- https://doi.org/10.5281/zenodo.23932258
- Primary Topic
- Industrial Vision Systems and Defect Detection
- Type
- preprint