OFT: An accelerator with eager gradient prediction for attention training

With the tremendous success of Transformer, resource-constrained edge devices are increasingly becoming the deployment target for attention-based models. On-device attention training can address the accuracy decline caused by static models' inability to adapt to dynamic environments efficiently while protecting data privacy. However, edge devices cannot meet the resource demands caused by batched gradient backpropagation and weight updates. Due to the inherent redundancy in human language, sparsification is the primary choice to alleviate the contradiction. Current sparse training methods consume high runtime costs in both forward propagation (FP) and backward propagation (BP) computations to handle irregular sparse patterns, which makes it inefficient to convert potential speedups into actual performance improvements and energy savings.