Capsule-Infused Vision Transformer: A Hybrid Deep Learning for Hierarchical Feature Learning

Diabetic Retinopathy (DR) is an eye condition that impairs the retina's blood vessels, leading to visual loss in diabetics. Microaneurysms are the early sign for DR and it is crucial for ophthalmologists to find manually and prevent vision loss. We proposed a hybrid model by concatenating the feature detected by capsule network and vision transformer. Before training the data, SMOTE method is used to balance the data sample and green channel image is taken for accurate detection of DR classes. The hybrid model leverages the strength of capsule network by preserving the spatial information by dynamic routing and vision transformer process the global image through self-attention mechanism. A classification accuracy of 98.1 % was attained when the suggested model was trained and evaluated on a sample of DR data. This proposed model automatically detects the early sign of diabetic retinopathy. The confusion matrix is demonstrated to show how the model's effectiveness is in correctly classifying different stages of DR.