A Unified Language Model Approach for the Semantic Interpretation of Traffic Signs

This work focuses on enhancing autonomous vehicle navigation systems by leveraging the semantic information contained in traffic signs, with a particular emphasis on text-based signs. We introduce an approach to interpret the text on signs for downstream autonomous vehicle applications, such as determining parking restrictions and identifying roadwork zones and directional commands. Specifically, we build upon our previously developed method [1] to create a more fine-grained representation of traffic sign parsing and utilize a pre-trained language model in a constrained setting. This approach helps alleviate the need for hand-crafted rules, making our method significantly more scalable. Our approach decomposes the parsing task into simpler tasks, each of which can be implemented using a single lightweight transformer-based language model, thereby improving efficiency and interpretability. We demonstrate the effectiveness of our method through experiments, showing substantial improvements in accuracy over a previously introduced baseline. To the best of our knowledge, this is the first application of a unified language model to this task, representing a significant step toward open-world sign interpretation.