NoisyNeighbor: Flatten Layer-Based Regularization in Convolutional Neural Network

Convolutional Neural Networks (CNNs) often overfit and generalize poorly to unseen samples. Existing regularization methods act on the input, hidden layers, or loss function, but overlook the flatten layer, which converts feature maps into a vector for the fully connected layers. Since this layer has no learnable parameters, memorized patterns pass through it unchanged. To address this gap, we propose NoisyNeighbor (NN), a regularization method that flattens the entire batch into a single vector and replaces randomly selected elements with the average of their neighbors, an inter sample perturbation that we formalize under the Vicinal Risk Minimization framework. NN adds no parameters and only negligible computational overhead. We evaluate NN on MNIST, FMNIST, SVHN, CIFAR-10, and CIFAR-100 with AlexNet and ResNet18, alone and combined with Dropout, Batch Normalization, Mixup, D-DropBlock, and ℓ 2 regularization. NN improves test accuracy and narrows the gap between training and test accuracy most on harder datasets with ResNet18, raising its accuracy on CIFAR-100 by 3.66 to 4.92 percentage points alone and with every method. On ResNet18, its gains are close to additive with Mixup, while it overlaps with Dropout on AlexNet. These results suggest that the flatten layer is a promising and inexpensive place for regularization.