Driving Like Humans: Leveraging Vision Large Language Models for Road Anomaly Detection

Ensuring the safety and reliability of autonomous driving systems requires accurate detection and response to road anomalies. Traditional object detection methods have advanced in identifying predefined objects, such as vehicles, pedestrians, etc., but often fail with unexpected or unknown obstacles. This paper introduces a novel approach using Vision Large Language Models (VLLMs) to enhance the adaptability of object detection through natural language interactions. By integrating visual data with human-like text prompts, our proposed approach dynamically adjusts to new and unforeseen obstacles. Through extensive evaluations, we demonstrate that our VLLM-based framework is comparable to the state-of-the-art road anomaly detection models, underscoring the potential of VLLMs in autonomous driving.