Disposable-key-based image encryption for collaborative learning of Vision Transformer

We propose a novel method for securely training the vision transformer (ViT) with sensitive data shared from multiple clients such as privacy-preserving federated learning. In the proposed method, training images are independently encrypted by each client where encryption keys can be prepared by each client, and ViT is trained by using these encrypted images for the first time. The method allows clients not only to dispose the keys but to also reduce the communications costs between a central server and clients. In image classification experiments, we verify the effectiveness of the proposed method on the CIFAR-10 dataset in terms of classification accuracy and the use of restricted random permutation matrices.