C2F-RMD: Automated Road Manhole Detection and Condition Assessment

Road manhole detection and damage assessment are crucial for ensuring the safety and efficiency of transportation systems. Traditional methods, reliant on costly and inaccessible three-dimensional (3D) cameras, pose challenges, especially in resource-limited settings. This study introduces C2F-RMD, a groundbreaking deep learning (DL)–based algorithm that revolutionizes road manhole detection and damage assessment using only two-dimensional (2D) images. C2F-RMD adopts a two-stage approach. In the first stage, the coarse-to-fine (C2F) detection technique, coupled with scale-adaptive region-based convolutional neural network (R-CNN), accurately detects and classifies road manholes. Achieving an impressive F1 score of 0.96 and an intersection over union of 0.95 across eight classes, the C2F model provides robust results. The second stage, road manhole damage (RMD) index, employs self-crack segmentation and an elevation prediction model. The self-crack segmentation, trained without labeled data, attains remarkable accuracy rates: 0.821 for precision, 0.805 for recall, and 0.813 for F1 score. The innovative elevation prediction model forecasts manhole surroundings’ elevation maps using solely 2D image input, with a regression score (R2) of 0.77 and a mean absolute error (MAE) of 4.35 mm. Notably, this method was successfully applied to an 802-km road network in Seoul City, encompassing various road types, including urban, principal, and supplementary roads, as well as expressways. It accurately detected and classified eight types of manholes with an accuracy rate of 0.98. Additionally, the method achieved accuracy rates of 0.80 for crack segmentation, 0.88 for crack segmentation grading, and 0.83 for elevation difference grading in manhole condition evaluation, demonstrating its adaptability in detecting, classifying, and evaluating manholes across diverse road types. This promising approach has the potential to replace traditional manual visual assessments of road manholes.