Damage Detection and Segmentation in Disaster Environments Using Combined YOLO and Deeplab

Building damage due to various causes occurs frequently and has risk factors that can cause additional collapses. However, it is difficult to accurately identify objects in complex structural sites because of inaccessible situations and image noise. In conventional approaches, close-up images have been used to detect and segment damage images such as cracks. In this study, the method of using a deep learning model is proposed for the rapid determination and analysis of multiple damage types, such as cracks and concrete rubble, in disaster sites. Through the proposed method, it is possible to perform analysis by receiving image information from a robot explorer instead of a human, and it is possible to detect and segment damage information even when the damaged point is photographed at a distance. To accomplish this goal, damage information is detected and segmented using YOLOv7 and Deeplabv2. Damage information is quickly detected through YOLOv7, and semantic segmentation is performed using Deeplabv2 based on the bounding box information obtained through YOLOv7. By using images with various resolutions and senses of distance for training, damage information can be effectively detected not only at short distances but also at long distances. When comparing the results, depending on how YOLOv7 and Deeplabv2 were used, they returned better scores than the comparison model, with a Recall of 0.731, Precision of 0.843, F1 of 0.770, and mIoU of 0.638, and had the lowest standard deviation.