Knowledge distillation and fine-tuning for corrosion and salt damage classification

This paper proposes a classification model based on a corrosion and salt damage dataset of civil structures: ViT is used as the teacher model, while MobileNetV2 and MobileNetV3 are used as student models. The second-best model is obtained by improving the classifier and setting different ratios of fine-tuning layers under frozen versus non-frozen classification layers. This model is added with an improved attention mechanism to get the optimal model. The final results show that using ViT with the second weight and MobileNetV2 gives higher Accuracy and Weighted-f1 value, where heat maps generated by applying Grad-CAM reflect that it can generally identify the damage location. The optimal model obtained by choosing a fine-tuning strategy that freezes the classification layers and retrains 15% of the top feature layers can achieve an Accuracy and Weighted-f1 score of above 0.94, better than many advanced deep learning architectures using pre-trained weights.