Predicting Future Driving Decisions in an Accident Situation From Videos: A Combined Behavioral, Eye Gaze, and Computational Analysis

Can human observers predict which of two directions a car will take in an accident situation when watching videos before the event recorded from the driver’s first-person perspective? And is it possible to use the observers’ eye-gaze data to predict their direction-choices? In our study with N = 30 participants, we first show that observers identify the correct direction already from 4 seconds(s) before the accident event with performance rising to 92% at 1s prior. Statistical analyses of the eye gaze data of the observers further identify patterns of gaze behaviors differentiating the observers’ choices. We then use an explainability approach to show that graph networks pay attention to similar scene parts as humans. Our results showcase the remarkable ability of human action predictions and that these predictions during complex, dynamic viewing can be classified from gaze data alone.