LVMScissor: Split and Schedule Large Vision Model Inference on Mobile Edges via Salp Swarm Algorithm

In the Computer Vision, the Large Vision Models (LVM) based on Vision Transformer (ViT) achieve advanced performance on general complex visual tasks. However, deploying these resource-intensive models on edge devices with limited computational power and memory is challenging, especially for those mobile edge devices. Existing model compression works downgrade the prediction accuracy and fail to adapt to dynamic network bandwidth or hardware changes. Besides, the split inference for typical Deep Neural Networks (DNN) is inefficient for LVM’s large intermediate result size. To address the computation and bandwidth limitation of edge devices of LVM, we design a new split inference acceleration LVMScissor by leveraging model parallelism and meta-heuristic algorithm. We first implement an inter-layer ViT parallelism strategy. After modeling the parallelized ViT split problem into a Multi-task three-processor scheduling (MTS) problem, we propose LVM-MSSA, a scheduling algorithm based on a famous meta-heuristic algorithm, the Multi-objective Salp Swarm Algorithm (MSSA) to schedule model parallelism and split strategy. The evaluation results show that compared to state-of-the-art inference acceleration approaches, our solution is faster from <inline-formula><tex-math notation="LaTeX">$1.6\times$</tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX">$6.92\times$</tex-math></inline-formula> under variant LVMs, edge devices, datasets, and network traces.