Patch Group-based Hide-and-Seek for Corruption Robustness of Vision Transformer

Various data augmentation techniques have been proposed in Convolutional Neural Networks (CNNs) to enhance their generalization ability. However, it remains unclear whether these techniques are equally beneficial in Vision Transformer (ViT). In this paper, we 1) identify the limitations that three augmentation methods−Random Erasing, Random Erasing in the Frequency domain (REF), and Hide-and-Seek−which have been proven effective in CNNs, fail to yield meaningful performance gains in ViT, and 2) propose a novel augmentation method, Patch Group-based Hide-and-Seek, which aligns with the architectural characteristics of ViT and improves robustness against corrupted input images. The main idea is to leverage the patch-based image processing of ViT by grouping spatially adjacent nx n patches within the image and randomly hiding each group. Experimental results using the ViT-B/16 model on CIFAR-10 and CIFAR-100 demonstrate that applying 3x3 patch grouping reduces corruption error by up to 0.335% and 0.220%, respectively, compared to the baseline. These findings highlight the effectiveness of patch-based data augmentation strategies in ViT and their contribution to improving robustness against corrupted datasets.