A Coupling Contrastive Learning with Biggest Margin Trees for Classifying Chest X-ray Images

This paper proposes a novel approach for improving chest X-ray image classification by integrating self-supervised representation learning with a new margin-based ensemble classifier. A major limitation of supervised learning in medical image analysis is its strong dependence on large labeled datasets, which are expensive and time-consuming to obtain. To address this issue, we leverage self-supervised learning to exploit abundant unlabeled X-ray images and learn informative visual representations before fine-tuning with limited labeled data. In addition, we introduce a new classification algorithm, termed Random Forest of Biggest Margin Trees (RF-BMT), designed to enhance multi-class classification by combining large-margin decision boundaries with ensemble learning. RF-BMT is trained on features extracted from a linear fine-tuned Momentum Contrast (MoCo) model with a ResNet50 backbone, and serves as an alternative to the conventional softmax classifier in deep networks. This design improves classification robustness and provides stronger generalization, particularly in limited-data settings. We evaluate the proposed approach on chest X-ray datasets containing multiple conditions, including normal, covid-19, edema, mass-nodule, and pneumothorax cases, under varying proportions of labeled training data. Experimental results demonstrate that the proposed method consistently outperforms standard classifiers and significantly improves the performance of the MoCo-based representation. In particular, RF-BMT achieves 86.2% accuracy using only 5% of the labeled training data (3,467 images), corresponding to a 7.4% improvement over the baseline, highlighting its effectiveness in limited-data scenarios. When trained with the full dataset, the model reaches 88.4% accuracy, demonstrating its robustness and scalability. These results show that combining self-supervised learning with the proposed RF-BMT classifier provides an effective and practical solution for medical image classification with limited labeled data.