Traffic Sign Recognition System with Hybrid CNN and Random Forest Models for Autonomous Driving Applications

Accurate traffic sign recognition is crucial for driver assistance systems, yet challenges persist due to environmental factors and camera distortions. This study introduces a novel framework for real-time traffic sign detection and classification using vehicle dash-cams. The system employs a fine-tuned YOLOv8 model for detection, achieving mAP50 of 99.348%. Classification utilizes a two-stage approach: VGG16-based image embedding generation followed by Random Forest classification, resulting in an F 1 score of 98.61%. The model is trained on a diverse dataset combining GTSRB and web-scraped images, with extensive augmentation to enhance robustness. Additionally, Microsoft’s TrOCR extracts numerical values from speed limit signs. This hybrid approach merges the strengths of convolutional neural networks and Random Forests, producing a robust solution even with a very limited amount of training data. The system’s high accuracy and real-time performance significantly enhance driver safety by ensuring reliable traffic sign recognition across various conditions.