FIOnA: Feature Invariant Data Augmentation for Small Datasets

Modern computer vision models face significant challenges when trained on small or domain-specific datasets, where large-scale data collection and labeling are impractical. This paper introduces FIOnA (Feature Invariant Online Augmentation), a novel data augmentation method grounded in the manifold assumption, which posits that images of the same class form a lower-dimensional manifold within a high-dimensional pixel space. FIOnA generates augmented samples by swapping semantically similar sub-features between images of the same class, and subsequently smoothing to ensure realism. We present a theoretical foundation that demonstrates the label-preserving nature of these feature exchanges. We also detail two featureextraction pipelines Neural Network and Transformer based, as well as two smoothing techniques Diffusion and Generative Adversarial Networks. Experimental evaluation on fine-grained image classification tasks shows FIOnA's potential to improve accuracy and generalization against baselines by 4.95 % and 2.02 % above state-of-the-art augmentation techniques, offering an alternative for data-scarce domains and proposing transformative research directions.