Gaze Direction Detection Using Dilated Convolutional Neural Network and Transfer Learning

In recent years, many eye-tracking related products are developed, focusing on eye movement and gaze tracking. The former mostly uses more expensive wearable or invasive devices, and the latter sometimes needs to utilize 3D feature points of the head, making data collection more cumbersome. Methods using pure RGB images as input are less common and often require a large amount of data. The direction of gaze is affected by many factors, including head posture as people tend to turn their heads in the direction they are looking. This paper proposes a two-stage gaze direction detection network which consists of the prediction of both head posture and gaze direction. The detection of head and eyes are first conducted, followed by the head posture estimation. We expand the receptive field to acquire more detailed pose information by employing the dilated convolution to HOPENet. For gaze direction prediction, dilated convolutional VGG16 is adopted for feature extraction. The head’s facing angle is then incorporated and passed through three fully connected layers to detect the gaze points of the eyes in the images. Due to the lack of public data for evaluation, in this work we create a new image dataset which utilizes an IMU to record the head posture and a projector-camera system to identify the gaze direction. Transfer learning is also employed to compensate for the limited dataset. The experiments using real-scene images have demonstrated the effectiveness of the proposed gaze direction detection technique.