Revisiting RIAWELC: Radiograph-Level Auditing Reveals Data Leakage and Class-Selective Domain Shift in Weld Defect Classification

Abstract Radiographic weld defect classification with deep learning has advanced through patch-based public datasets, but patch-level evaluation can produce optimistic results when patches from the same source radiograph appear in both training and testing sets. We revisit the RIAWELC dataset. An MD5 audit confirms all 2,443 files in the original test set are byte-identical duplicates of training files, consistent with an independent, concurrent report; previously reported accuracies therefore measure memorization rather than generalization. We construct RIAWELC-RG, a radiograph-level split that removes all duplicates and assigns every patch from a given source radiograph to a single subset. Because leakage correction alone does not reveal domain shift within individual source radiographs, our main contribution is a radiograph-level auditing protocol combining per-radiograph performance analysis, embedding-distance comparison, statistical intensity pro filing, and domain-classifier attribution, demonstrated on bam5, an anomalously named radiograph in the test set. Under a fully fine-tuned ConvNeXtV2-T, degra dation from bam5 is class-selective: crack accuracy drops to 0.380 while lack of penetration and porosity remain at 1.000. A statistical-feature domain clas sifier separates bam5 from RRT-domain radiographs almost perfectly (Random Forest area under the receiver operating characteristic curve of 0.9995), with Laplacian sharpness variance the strongest factor, implicating acquisition blur. The pattern reproduces under 5-fold radiograph-level cross-validation and across thirteen independently trained convolutional (CNN) and Transformer architec tures, all showing a positive accuracy gap when bam5 is included (paired t-test, p < 10−6).Without radiograph-level evaluation and domain-shift auditing, classifiers trained on datasets like RIAWELC risk mistaking acquisition artifacts for genuine generalization.