Vision‐Language Hazard Reasoning for Driver Distraction and Workload Estimation

ABSTRACT Driver distraction and cognitive overload are major contributors to road accidents, especially in semi‐automated driving where humans must remain situationally aware. We propose an agentic reasoning approach for jointly detecting external driving hazards and inferring internal driver workload, such as distraction and cognitive overload, through a vision‐language‐model (VLM)‐centric framework. The proposed framework interprets driving scenes alongside customisable natural language‐based safety rules, enabling context‐sensitive hazard reasoning. It classifies scene complexity as a proxy for cognitive task demand and selectively invokes multimodal tools to further refine risk assessment based on task complexity or uncertainty. As a pilot study, we evaluate the framework's capacity for hazard comprehension using a curated set of driving scene images, sampled across five rule domains: driver distraction, traffic rules, pedestrian behaviour, road condition and vehicle load safety. Preliminary results demonstrate that the VLM‐based method can correctly identify safety rule violations and provide interpretable rationales for explaining cases.