Toward Automatic Pavement-Condition Index Estimation: An Enhanced Multi-Sensor Deep Learning Framework for Comprehensive Pavement Distress Detection and Severity Assessment

Timely and accurate pavement distress detection is essential for sustainable road maintenance. Traditional manual methods are costly, time-consuming, and prone to subjectivity. This study proposes a multi-sensor framework that integrates (Red-Green-Blue (RGB) imagery with low-cost depth sensing for automated detection, classification, and severity quantification of pavement distresses. The system identifies and categorizes 11 common distress types using convolutional neural networks and a sensor fusion strategy. Detected regions are projected onto three-dimensional point clouds to enable class-specific severity assessment. Pothole and rutting severity classification are examined as case studies. Using a low-cost Red Green Blue plus Depth (RGB-D) multi-sensor fusion framework, we report, for the first time, automated rutting severity classification integrated into Pavement-Condition Index (PCI) estimation as a proof-of-concept, alongside enhanced pothole severity quantification and classification. Results show high accuracy, with pothole severity correctly estimated in 96.2% of cases and rutting severity classified with 100% accuracy. The system also reliably detects shallow potholes as small as 15 mm in depth. By automating key measurements required for PCI estimation, this work lays a practical foundation for data-driven pavement evaluation. The proposed approach offers a scalable and cost-effective solution for infrastructure monitoring, reducing reliance on manual inspection and enhancing roadway management, while also demonstrating extensibility to other distress types.