SISEVIR: From Manual Inspection to Automated Diagnosis of Vertical Traffic Signs Through YOLO Segmentation, EfficientNet, and Vision–Language Models for National Road Safety Management in Peru

Inventory and condition assessment of vertical traffic signs constitutes an essential activity for road safety management, as these signs serve as the primary mechanism for regulating vehicular flow and alerting drivers to prevailing road conditions. However, current inspection procedures in Peru rely on field crews that evaluate each sign manually, thereby constraining the frequency, objectivity, and scalability of the process. This paper presents SISEVIR (Sistema de Supervisión de Señales Verticales en Infraestructura Vial), a three-stage deep learning pipeline for the automated diagnosis of vertical traffic sign condition. The first stage employs YOLO26s-seg for instance segmentation of 31 sign classes, achieving a test mAP50 of 0.9305 (box) and 0.9224 (mask). The second stage classifies each detected sign into seven deterioration states using EfficientNet-B0, optimized through a five-experiment ablation study that identified progressive offline augmentation as the most effective strategy for handling a 147:1 class imbalance (macro F1 = 0.8252; pairwise McNemar’s tests with Holm–Bonferroni correction did not confirm significance at the family-wise α=0.05 level). The third stage integrates Qwen2-VL-2B-Instruct, a vision–language model, to generate natural-language descriptions of sign condition aligned with the MTC Manual of Traffic Control Devices for Streets and Highways. A structured evaluation by two independent raters on 35 descriptions yielded a correctness rate of 93.5% among valid responses (95% CI: 79.3–98.2%, Cohen’s κ=1.00). The system was trained and validated on a proprietary dataset of 5935 images and 6412 labeled crops collected along three routes in the Ayacucho Region (246.6 km total), with an inter-rater reliability of κ=0.802 (95% CI: 0.676–0.928). SISEVIR processes vehicular video at 30.7 FPS on an NVIDIA RTX 5080 GPU and assigns each sign a level within a four-tier condition scale (Optimal through Critical) linked to specific maintenance interventions, significantly reducing the time, cost, and personnel required compared with the manual inspection method established in the MSV-2016 Road Safety Manual.