Vulnerability analysis of transformer-based optical character recognition to adversarial attacks

We present a novel framework to assess the resilience of state-of-the-art transformer-based optical character recognition (TrOCR) models. In this way, we develop new untargeted and targeted attack algorithms. On a benchmark handwriting dataset, we show that adversarial perturbations to the input reveal dramatic vulnerabilities without being noticeable to the eye. This article is part of the theme issue 'Safe, secure and robust AI for safety-critical systems'.