Analyzing and Enhancing the Reliability of Vision Transformer Models Against Soft Errors

Vision Transformer models have become a powerful class of deep learning models, excelling in various computer vision tasks. Their integration into safety-critical systems necessitates a robust understanding of their reliability. This is particularly crucial due to the rising vulnerability of modern computing systems to transient faults, namely soft errors. In this paper, we analyze the reliability of Vision Transformer models in the presence of soft errors by employing parameter fault injection. We also propose a simple yet highly effective technique against soft errors, which can enhance the reliability by restoring the Top-1 accuracy of Vision Transformer models on the ImageNet-1K dataset to nearly original levels.