ETDH: Efficient Transformer Inference with Dynamic Head Selection in Collaborative Edge Environments

Transformer and its large variants are widely used in Natural Language Processing and Computer Vision, which bring the high computational cost and delay in reasoning phase, and become the key bottleneck of edge device deployment. Collaborative edge computing provides new opportunities for large model local reasoning, but the differences between heterogeneous devices and the redundancy of multi-head self-attention mechanism still need to be solved. This paper proposes an ETDH framework, which is oriented to the collaborative heterogeneous edge environment. Through head level parallelism and dynamic head selection, combined with the task allocation of device power perception and the head number adjustment driven by sample characteristics, it realizes load balancing and redundancy reduction. Experiments show that, on edge platforms such as NVIDIA Jetson Orin nano and Xavier NX, ETDH has significantly better reasoning delay than the mainstream methods with near lossless accuracy, which provides theoretical and engineering support for the efficient deployment of transformer in edge scenes.