Detecting and Locating Multiple Pavement Distresses Using Weakly Supervised Learning Methods

Pavement management and rehabilitation systems require reliable detection and assessment of road pavement distresses. Traditional methods that rely on visual inspections and manual surveys are time-consuming and inconsistent. Several computer vision techniques for distress detection automation have been developed. Most techniques are based on fully supervised labeling, which is especially challenging with respect to locating the exact position of a distress in an image. This paper introduces an innovative approach combining machine learning and image analysis techniques for the automated classification and localization of multiple pavement distresses. In contrast to previous approaches that detected a single distress per image, the proposed approach can both classify and locate multiple distress types per image without prior knowledge of the distress’ location. The system classifies multiple distress types within a single image, using a multilabel convolutional neural network (CNN). Distress localization is enhanced through an interpretability method, namely, integrated gradients, followed by postprocessing denoising techniques such as dilatation and connected components. The proposed method is applied in a large-scale dataset collected as part of the pavement management system, which contains information about the number and type of distresses, but not their location in the image. Results indicate that the multilabel CNN model for classifying distress types reached a precision of 84.41%, and the proposed localization method reached 77% overall accuracy, 63.3% for images with multiple distress types, and 82.9% for images with a single distress. The method localizes line distress types more precisely than area distress types. The selected postprocessing parameters critically affect the localization precision.