Enhancing Split ViT Inference Through Sparsity-Driven Compression

Vision Transformer (ViT) is a deep learning model that plays a significant role in advanced computer vision and pattern recognition tasks but faces challenges in inference due to high computational costs and energy consumption. Split computing, which distributes the load between an edge server and a mobile device, is proposed to address these issues. Unfortunately, most literature focuses on split computing for CNNs, with limited attention to ViT split computing. In this paper, we explore the use of split computing to optimize ViT's inference process by limiting usage of bandwidth. We propose compressing the latent space data by introducing more sparsity in the intermediate features. This sparsity is then exploited through a compression algorithm before transmitting the data through a communication channel. To understand and explain the performance of our approach, we analyze the latent space data using several metrics. Our approach obtains a significant compression ratio without causing a substantial decrease in accuracy. It is computationally efficient and do not require retraining the model.