Evaluation of data collection and annotation approaches of driver gaze dataset

Driver gaze estimation is important for various driver gaze applications such as building advanced driving assistance systems and understanding driver gaze behavior. Gaze estimation in terms of gaze zone classification requires large-scale labeled data for supervised machine learning and deep learning-based models. In this study, we collected a driver gaze dataset and annotated it using three annotation approaches - manual annotation, Speak2Label, and moving pointer-based annotation. Moving pointer-based annotation was introduced as a new data annotation approach inspired by screen-based gaze data collection. For each data collection approach, ground truth labels were obtained using an eye tracker. The proposed moving pointer-based approach was found to achieve higher accuracy compared to the other two approaches. Due to the lower accuracy of manual annotation and the Speak2Label method, we performed a detailed analysis of these two annotation approaches to understand the reasons for the misclassification. A confusion matrix was also plotted to compare the manually assigned gaze labels with the ground truth labels. This was followed by misclassification analysis, two-sample t-test-based analysis to understand if head pose and pupil position of driver influence the misclassification by the annotators. In Speak2Label, misclassification was observed due to a lag between the speech and gaze time series, which can be visualized in the graph and cross-correlation analysis were done to compute the maximum lag between the two time series. Finally, we created a benchmark Eye Tracker-based Driver Gaze Dataset (ET-DGaze) that consists of the driver's face images and corresponding gaze labels obtained from the eye tracker.