Enhancing Transmission Efficiency in Vision Transformer Edge Computing via Token Clustering

Vision Transformer demonstrates excellent performance, but huge number of parameters hindered deployment. Offloading workloads to the cloud is a promising solution. However, transmission latency and bandwidth limit the efficiency of cloud. Edge computing technology is used to mitigate this problem. Leveraging the computation power of edge devices, data is processed before uploading to reduce transmission latency. Therefore, this paper proposes a clustering-based token subsampling method for the vision transformer partitioning to enhance transmission efficiency. Tokens are encoded based on the cosine similarity clustering result, then all tokens are reconstructed at cloud. Experimental results show that the proposal is effective and can be executed with low latency. For ViT, 23.09% transmitted data is reduced with an accuracy drop of 0.44% within 0.8 ms.