Flexible Transformer: A Simple Novel Transformer-based Network for Image Classification in Variant Input Image Sizes

Convolutional neural networks (CNNs), the most important deep learning networks for computer vision, have undergone a series of developments and improvements for image-related tasks such as object recognition, image classification, semantic segmentation, etc. However, in the field of natural language processing (NLP), the novel attention-based network Transformer had a profound impact on machine translation, which subsequently led to a boom in attention-based models for computer vision. State-of-the-art models with attention have already shown good performance for computer vision tasks due to the sophisticated design of network architectures and advanced computational efficiency techniques. For example, selfattention learns relationships between segments or words in different positions compared to the current performance of convolutional neural networks. Inspired by Vision Transformer (ViT), we propose a simple novel transformer architecture model, called Flexible Transformer, which inherits the properties of attention-based architectures and is flexible for inputs of arbitrary size. Besides self-attention, the inputs in ViT are not pre-processed, such as resizing or cropping, but kept intact without altering them, which could lead to distortion or loss of information. In this paper, we want to present a novel and simple architecture that meets these requirements. Compared to state-of-arts, our model processes inputs with arbitrary image sizes without any pre-processing and pretraining costs. Also, the results of the experiments show that the model can potentially provide good results with high accuracy despite limited resources. Even though the results of Flexible Transformer are not as accurate as those of Vision Transformer, they show the potential of a model with high performance in image classification tasks with variable size images. The significance of the research opens up possibilities for dealing with primitive images in deep learning tasks. Based on the original inputs, reliable results with good accuracy could be obtained if the proposed model is optimized and further trained on large datasets.