Integrating Vision Transformers with Self-Supervised Learning and Contrastive Learning for Lung Disease Classification

Lung disease classification is a critical task in medical imaging that requires accurate and robust feature extraction methods. Traditional Convolutional Neural Networks suffer from limited receptive fields, making them less effective in capturing long-range dependencies and global contextual features in medical images. To overcome these limitations, this study proposes a Vision Transformer-based model enhanced with Self-Supervised Learning and Contrastive Learning for automated lung disease classification. The proposed method first employs masked auto-encoders to learn meaningful feature representations from unlabelled medical images, reducing the dependency on large labelled datasets. Contrastive Learning, ensuring distinct class separability by bringing similar representations closer and pushing dissimilar ones apart. After pre-training with SSL and CL, the model is fine-tuned with supervised learning using a fully connected classification refines these learned features head. The experimental results demonstrate that the proposed approach achieves higher classification accuracy compared to standard deep learning models, including ResNet50, DenseNet121, and InceptionV3. This research highlights the advantages of self-supervised pre-training in medical imaging and showcases the potential of transformer-based architectures for clinical applications.