A didactic overview of Transformer applications: model variations and user guidelines

Abstract Transformers models were originally designed for the processing of textual data. In the last years they have been extended to handle different modalities of data including image and video, audio, tabular, and even multimodal data. Adapting the vanilla Transformer architecture is necessary to optimize the performance for each data type. The vast amount of new architectures that have emerged makes it difficult to detect and understand the differences with the original Transformer. This paper provides an overview of Transformer applications for various input modalities, and recommendations to guide the development and use of models.