Augmenting the Perceptual Experience of Being Faced Using Gaze Modeling during Online Video Viewing

Live Performances have been conducted online in recent years due to the influence of the COVID-19 pandemic. This generated a novel problem of losing eye contact with the performers, which deteriorates the viewing experience in the online setting. This study proposes a feedback system based on gaze modeling to augment the perceptual experience of being faced by the performer during an online live performance viewing. This system aims to improve the experience of viewing online live performances by recreating a perceptual experience of interaction between the performer and the audience. The proposed system consists of a gaze recognition unit that judges whether the performer is looking at the camera based on gaze modeling in the video, and a stimulus presentation unit that presents the judgement result to the audience through vibration stimulus. This paper describes the gaze recognition model, experiments to determine the parameters to be used in the model, and the evaluation of live performance viewing experiments using the proposed system.