What is the best way to merge multiple images from recorded video for diabetes classification using Vision Transformer?

Vision transformers (ViT) perform remarkably on large datasets but often underperform compared to convolutional neural networks (CNNs) when trained from scratch on smaller datasets. This study aimed to improve the accuracy of ViT in diabetic patient classification with the nail-fold capillaries dataset. It involved preprocessing images to extract relevant features and, unlike previous methods that used a single image, it combined multiple images into a composite for simultaneous analysis of video frames. The results demonstrated a substantial 27.53% improvement, with the F1 score increasing from 32.26% to 59.79% when using the proposed model training from scratch. Additionally, the F1 score also increased by 3.93% and achieved 98.56% with the pre-trained model using our proposed method which integrated multiple nail fold capillaroscopy images and employed ViT. Detailed analysis revealed that merging the nine images into a 3×3 format instead of a panoramic format yielded optimal results.