CO-Net: Multi-stream Gaze Estimation Model in Unconstrained Environments

The eyes are the windows for humans to interact with the outside world and a prerequisite for the promotion of many applications. In gaze estimation, although the full-face image encompasses the eyes, when extracting gaze features, the network often comprehensively considers the information characteristics of the entire face and its interrelationships. This may lead to neglecting the delicate gaze-related information contained in the eye images. To improve the accuracy of gaze direction prediction, this study proposes a novel multi-stream binocular gaze estimation model named CO-Net, which takes facial and binocular images as input. First, we use facial and eye features to preliminarily estimate a primary gaze direction. Considering the differences that arise when observing objects with both eyes in an unconstrained environment, we utilize this characteristic to purify the gaze features. Finally, we use the purified features to linearly optimize the primary gaze direction. Experimental results demonstrate that our method reduces the mean angular error to 3.63 ∘ on MPIIGaze and 5.9 ∘ on RT-GENE, representing improvements of 7% and 10%, respectively, over the previous state-of-the-art.