Leveraging Visual Language Models for Detecting Distracted Driving

Distracted driving is a leading cause of road crashes, yet traditional analysis often relies on labour-intensive manual video annotation. This study investigates the use of the Vision-Language Model (VLM) PaliGemma 2 to automatically detect driver distraction. Using video data from the Australian Naturalistic Driving Study (ANDS), which captures the driver's steering wheel, we sampled frames and processed them through PaliGemma 2 to generate textual descriptions of behaviour. The model was fine-tuned by updating only its attention layers, improving recognition of distractions like phone use and adjusting controls while retaining general knowledge. Outputs were standardised to enable systematic analysis. Preliminary results show that VLM-based detection accurately identifies key distraction behaviours, greatly reducing the need for manual labelling. These findings support the use of VLMs in road safety research, driver monitoring, and AI-driven interventions, and mark one of the first applications of large-scale multimodal models to naturalistic driving data.