Hybrid Computer Vision-Based and Multidefect Data Fusion Framework for AI-Enhanced Concrete Bridge Deck Condition Rating

Abstract In recent years, computer vision (CV) methods have been increasingly explored for the detection of bridge defects. However, existing approaches generally lack an automated and standardized framework capable of consistently handling multiple defect types, which leads to variability in condition assessment outcomes. Moreover, current studies often suffer from limited multidefect data fusion capabilities, a heavy reliance on large, manually annotated data sets, and insufficient transparency in classifier decision-making processes. To address these limitations, this study introduces a hybrid CV-based methodology for concrete bridge deck condition rating. The methodology employs the YOLOv11 algorithm to detect and evaluate concrete bridge deck defects and integrates defect-specific severity quantification with multidefect data fusion for a more accurate and reliable condition rating. Additionally, the approach leverages transfer learning from high-accuracy models developed on Roboflow 2.0, thereby reducing dependence on extensive annotated data sets. Defect severity is quantified using domain-specific techniques aligned with National Bridge Inventory (NBI) inspection guidelines, and decision-level data fusion is implemented through rule-based logic to establish empirically grounded condition thresholds. To test this methodology, multidefect images from 20 bridges were analyzed using data sourced from bridge inspection reports and NBI manuals. Quantitative evaluation shows that the full hybrid framework achieves low prediction error in NBI condition ratings, with a mean absolute error (MAE) of 0.40 NBI units and a root mean squared error (RMSE) of 0.89 NBI units under the evaluated inspection conditions. An ablation study further demonstrates that defect severity quantification and decision-level data fusion contribute to progressive performance improvements relative to detection-only and quantification-only model variants. Performance improvements are attributed to the integration of grayscale-based image processing for leaching assessment and polygon-based geometric metrics using the Shoelace algorithm for crack severity evaluation, as well as explicit rule-based fusion of multiple defect types. Overall, the proposed methodology automates the inspection process by leveraging hybrid CV, multidefect data fusion, and NBI guidelines, ensuring consistent, objective, and efficient assessments, paving the way for advanced predictive maintenance strategies.