Explainable Image Captioning for Autonomous Driving: a Traffic Sign Recognition Task

Concentrating on identifying traffic signals and captioning images, this document emphasizes the essential need for making autonomous driving systems more Explainable. Utilizing a custom dataset of 4,800 traffic sign images, we present a hybrid approach that merges advanced image feature extraction methods (ResNet50, ExperientNetB0, InceptionV3, VGG16) with text-creation strategies (GRU, LSTM). An integrated dual attention mechanism amplifies Explainability by highlighting relevant visual indicators and key text components. The analysis, employing measurements like BLEU, ROUGE, and METEOR, shows significant improvements in the accuracy of describing traffic signs and the overall Explainability of the model, leading to improved safety in autonomous driving.