Development of an emotion prediction system combining eye tracking and blink detection

Eye tracking technology is recognized increasingly for its ability to capture nonverbal emotional cues by analyzing eye movements and gaze patterns. However, traditional eye tracking systems are often expensive and have limitations in terms of usability. Additionally, blink patterns are key indicators of emotional and psychological states, but current systems fail to incorporate blink detection effectively. This paper proposes a cost-effective system that combines eye tracking and blink detection using a standard webcam to estimate psychological states. The system uses biometric information such as changes in eye gaze and blink frequency to provide a more comprehensive analysis of psychological states. This study introduces a new method consisting of face and eye feature point extraction, gaze tracking, and blink detection using the Eye Aspect Ratio (EAR) to evaluate the state of eye open and closed. For gaze position estimation, GazeNet pre-trained on the MPIIGaze dataset and along with individual calibration data are used. Experimental results demonstrate that estimation accuracy is improved significantly by calibration, and that spatial factors, such as the relationship between the camera and the display screen, influence performance. The results of these experiments suggest that the integration of blink detection and eye tracking in this system could contribute to the prediction of emotional and psychological states. Future works will focus on further validating the system’s reliability and its potential for emotion prediction.