Naturalistic driving data extraction and processing for studying driver head scanning behavior at intersections

Naturalistic driving (ND) studies offer key insights about real-world driver behavior compared to lab and on-road studies along a fixed course. We describe an ND data processing pipeline for characterizing the scene and driver behavior at intersections. The in-car recording system recorded the vehicle’s speed, location, and videos of the cabin and the front scene. Intersection locations were approximately marked from the drive log, and corresponding cabin and scene videos were clipped for ± 100 m segments around the marked intersection locations. Scene videos were processed to extract intersection scene characteristics including the type of signage, vehicle maneuver, traffic density, and to precisely estimate the vehicle’s entry and exit from an intersection. Driver head pose in cabin videos was estimated using a custom-developed AI model for wide angle head turns. Evaluation involved 365 manually-annotated intersection instances from 6 vehicles driven in 3 US states. Signage and maneuvers were correctly detected in 99% and 95% of cases, respectively. Median[IQR] intersection entry error was 1.1[0.3–4.2] meters, 0.23 [0.07–0.63] seconds; overlap between estimated and ground truth intersection bounds was 0.88[0.8–0.94]. There was strong positive correlation between estimated and annotated traffic density (R = 0.73, p < 0.001). Head pose estimation over a range of ± 135° had a mean absolute error of 6.75 ± 5.0° and an RMSE of 8.4°. Our approach can potentially allow automated processing of a vast amount of ND data while enabling the study of specific driver behaviors that were previously not achieved.