Adversarial Cueing Model Based on Vision Transformer and Nuclear Wasserstein Discrepancy

Deep neural networks require extensive of well-labeled training data to achieve optimal performance. However, real-world datasets are often entirely unlabeled. At the same time, the problem of feature distribution discrepancies prevalent in datasets can also lead to performance degradation of the model when performing data migration, posing significant challenges for practical applications. We first introduce a framework based on generative adversarial networks that improves the model's adaptability by reducing feature distribution discrepancies. Second, we freeze most of the Vision Transformer (ViT) parameters and introduce a small number of learnable parameters as training parameters in each layer of ViT to ensure further finetuning without destroying the structure of ViT. Last, we also combines the original discriminator with Nuclear Wasserstein Discrepancy(NWD), and the new discriminator satisfying the K-Lipschitz constraints obtained is clear guidance for performance discrimination. Compared with six other domain adaptation methods, our model achieved an average improvement of 6.35 percentage points on the Office31 dataset and 0.3 percentage points on the Visda-2017 dataset.