Nanoscale Computing: Assessment of Hardware Platforms for Machine Learning by Transformer Models

The performance in terms of execution time of Transformer-based models when run on edge-computing platforms is hard to predict, because it depends on many factors such as the nanoscale technology used for implementation or model architecture. Another dimension of interest when evaluating the execution time is the size of the model, as in many cases, several versions of the model with different sizes can be used. The smaller size versions reduce significantly the number of parameters and thus of the nanoscale memory and arithmetic operations needed to run the model, however this does not necessarily map to similar reductions in execution times. This paper studies the relation between model size and execution time by evaluating three different Transformer-based models: Bidirectional Encoder Representations from Transformer (BERT), Transfer Text-to-Text Transformer (T5) and Contrastive Language–Image Pre-training model (CLIP), and their corresponding reduced versions when run on three different nanoscale hardware platforms.