Measuring the Effects of Transfer Learning for CNN and Transformer Based Network on Skin Disease Classification

The rapid advancement in the study of machine learning have allowed for it use in multiple areas, with this demand comes the demand faster and more accurate models, however the high quality cleanly labeled data for use in machine learning is still a rarity as most data is either fuzzy or less precise with its labeling. However, Transfer Learning has emerged as a valuable tool for use in such conditions by adapting pre-trained models for different tasks. Our study highlights the significant impact of transfer learning on model accuracy, it shows that approximately a 24% improvement using pre-trained weights compared to baseline from scratch models, when combined with GradCam-IoU this further collaborates that the improved generalization capabilities allow the model to have higher accuracies. In the future we will focus more on the analysis from GradCam-IoU for multiple types of data.