Augmentation-induced sibling leakage in small-data surface imaging: measurement and a reporting protocol

Abstract Deep learning papers on surface and microscopy imaging routinely build their training sets by augmenting a handful of acquisitions into hundreds or thousands of images, and then split the augmented pool at random. Every test image then has a near-copy in the training set, and the reported metric measures recognition rather than generalisation. We quantify the cost. Holding the annotation budget fixed and varying only the provenance of the training images, we compare a random split (A) with a split by acquisition (B) on two public benchmarks and two tasks: surface-defect detection with Faster R-CNN, and point-defect segmentation in scanning tunnelling microscopy with a U-Net. Random splitting overstates [email protected]:0.95 by 0.546 at two source images per class and by 0.303 at ten; on the STM data it overstates Dice by 0.278--0.448. At the smallest budgets the random split drives [email protected] to 1.000 $\pm$ 0.000, a number that describes the test set rather than the model. The gap narrows as acquisitions increase; what declines is the benefit of leaking, not the quality of grouped evaluation. We propose a four-item reporting protocol and a sibling-leakage statistic that costs seconds to compute and whose error behaviour we characterise.