Enhancement Framework for Vision Transformers in Data Limited Cancer Detection

Vision Transformers (ViTs) have demonstrated the potential to outperform Convolutional Neural Networks (CNNs) on various imaging tasks. However, ViTs necessitate large training datasets to achieve comparable results, limiting their practicality in medical applications where high-quality data collection is limited. In this paper, I present a ViT modification framework, exploring previous approaches. I also propose using PCSVI (Post-Convolutional Single Vector Input) and field specification guidance during pre-training. It outperformed vanilla ViT performance on three scaled-down datasets by an average of $\mathbf{2 7 \%}$, achieving similar results to traditional CNN models. This framework can popularize ViTs in future AI medical research if applied carefully.