Estimating the repeatability and reproducibility of an AI-embedded measuring device: Application to road markings

The road markings inspection using on-board cameras is gaining popularity, particularly for road inspection, despite most research focusing on autonomous vehicles. Traditional inspection methods rely on costly equipment like RetroTek-D, resulting in infrequent inspections. Recent advancements in artificial intelligence offer a more cost-effective solution, enabling the use of standard cameras with AI models for continuous road marking detection. While metrics such as F1-score evaluate AI performance, they do not ensure effectiveness on new sites or account for uncertainty in AI applications. Once deployed, AI systems may not adapt to new data, necessitating retraining for any updates. This paper proposes a method to estimate the repeatability and reproducibility of AI-based measurement devices, drawing inspiration from ISO 5725 standards, to enhance their reliability in road marking detection.