Accelerated ElasticTrainer With Elastic Layer Selection

On-device training consumes a lot of training time due to the limited computing resources of edge devices. ElasticTrainer reduces training time by selecting important tensors from the model and then training them. However, selection at the tensor level leads to reduced arithmetic intensity, failing to fully utilize GPU resources. In this paper, we propose a layer-level selection method considering arithmetic intensity to further reduce training time. Compared to the existing tensor selection method, ElasticTrainer, our method reduces training time by up to 25% with less than 0.1% accuracy loss.