Bridge Surface Defect Localization Based on Panoramic Image Generation and Deep Learning-Assisted Detection Method

Applying unmanned aerial vehicles (UAVs) and vision-based analysis methods to detect bridge surface damage significantly improves inspection efficiency, but the existing techniques have difficulty in accurately locating damage, making it difficult to use the results to assess a bridge’s degree of deterioration. Therefore, this study proposes a method to generate panoramic bridge surface images using multi-view images captured by UAVs, in order to automatically identify and locate damage. The main contributions are as follows: (1) We propose a UAV-based image-capturing method for various bridge sections to collect close-range, multi-angle, and overlapping images of the surface; (2) we propose a 3D reconstruction method based on multi-view images to reconstruct a textured bridge model, through which an ultra-high resolution panoramic unfolded image of the bridge surface can be obtained by projecting from multiple angles; (3) we applied the Swin Transformer to optimize the YOLOv8 network and improve the detection accuracy of small-scale damages based on the established bridge damage dataset and employed sliding window segmentation to detect damage in the ultra-high resolution panoramic image. The proposed method was applied to detect surface damage on a three-span concrete bridge. The results indicate that this method automatically generates panoramic images of the bridge bottom, deck, and sides with hundreds of millions of pixels and recognizes damage in the panoramas. In addition, the damage detection accuracy reached 98.7%, which is improved by 13.6% when compared with the original network.