Enhance the Concrete Crack Classification Based on a Novel Multi-Stage YOLOV10-ViT Framework

Early identification of concrete cracks and multi-class detection can help to avoid future deformation or collapse in concrete structures. Available traditional detection and methodologies require enormous effort and time. To overcome such difficulties, current vision-based deep learning models can effectively detect and classify various concrete cracks. This study introduces a novel multi-stage deep learning framework for crack detection and type classification. First, the recently developed YOLOV10 model is trained to detect possible defective regions in concrete images. After that, a modified vision transformer (ViT) model is trained to classify concrete images into three main types: normal, simple cracks, and multi-branched cracks. The evaluation process includes feeding concrete test images into the trained YOLOV10 model, identifying the possible defect regions, and finally delivering the detected regions into the trained ViT model, which decides the appropriate crack type of those detected regions. Experiments are conducted using the individual ViT model and the proposed multi-stage framework. To improve the generation ability, multi-source datasets of concrete structures are used. For the classification part, a concrete crack dataset consisting of 12,000 images of three classes is utilized, while for the detection part, a dataset composed of various materials from historical buildings containing 1116 concrete images with their corresponding bounding boxes, is utilized. Results prove that the proposed multi-stage model accurately classifies crack types with 90.67% precision, 90.03% recall, and 90.34% F1-score. The results also show that the proposed model outperforms the individual classification model by 10.9%, 19.99%, and 19.2% for precision, recall, and F1-score, respectively. The proposed multi-stage YOLOV10-ViT model can be integrated into the construction systems which are based on crack materials to obtain early warning of possible future deformation in concrete structures.