Dataset Explanation Signatures for Visual Recognition: A Model-Free Region-Level Reference for Shortcut and Label-Quality Auditing

Visual recognition models are usually accepted on the basis of held-out accuracy, which does not reveal whether a model relies on image regions that actually carry label information in the training data. Post-hoc saliency methods describe a trained network, so they cannot be used to decide, before training, which regions a well-behaved model ought to use. We present Dataset Explanation Signatures (DES) for images: a region-level reference computed from a labelled image set alone, with no fitted predictor. For each of a set of fixed anchor images, DES compares the label impurity of the neighborhood formed in the full pixel space with the neighborhood formed after one image region is removed from the distance computation, and summarises the resulting field by its mean, dispersion, and rank stability under bootstrap resampling and pixel perturbation. On Fashion-MNIST and the scikit-learn handwritten digits benchmark, signature estimation takes about six seconds on a single CPU core and yields region maps whose ordering agrees moderately with the occlusion maps of five trained model families (map-level rank correlation 0.08 to 0.39 and 0.73 to 0.79, respectively), while agreement at individual anchors is weak. In a controlled spurious-cue study, DES ranked an injected corner marker first in every run once the marker’s correlation with the label reached 0.8, and it did so before any model had been trained. Models subsequently trained on the same data lost between 6.8 and 41.6 percentage points in accuracy when the marker was randomized at test time. As the rate of symmetric label corruption increased, both rank stability and positive contribution mass tended to decline overall.