An efficient 2D-3D fusion method for bridge damage detection under complex backgrounds with imbalanced training data

Existing bridge structures are inevitably affected by various adverse environments and loads during routine operations, which accelerates structural damage and highlights the necessity of conducting bridge inspections. Because of their cost-effectiveness and non-contact capabilities, computer vision methods applied to images from unmanned aerial vehicle (UAV) survey campaigns are promising ways to conduct bridge inspections. Bridge images captured by UAVs often contain numerous complex background pixels due to the small size of damage. Additionally, the existing damage datasets used for training suffer from a severe inter-class imbalance, which significantly affects the accuracy of damage recognition. This study proposes a 2D-3D fusion method for bridge damage segmentation and localization, effectively identifying damage under complex backgrounds with imbalanced data. First, a 3D reconstruction method is introduced to reconstruct bridge point clouds and generate depth maps from different viewpoints. Second, an RGB-D segmentation model is presented to extract the region of interest from images by integrating 2D and 3D information. Third, an improved DeepLabv3 + model is developed to segment damage and integrate it with point clouds for three-dimensional visualization. Field experiments are conducted on a multi-span simply supported girder bridge to validate the effectiveness of the proposed method. The ROI extraction model achieves an F-measure of 98.85%, and the damage segmentation model attains a mAP of 82.21%. Additionally, the 3D visualization result indicates areas of interest (e.g., wet spot, cavities, and spalling) on the cover girder, providing valuable guidance for bridge maintenance. These findings demonstrate the effectiveness and practicality of the proposed method in bridge inspection.