Face Tracking using Deep Reinforcement Learning on Zenbo Junior

This paper aims to develop a deep reinforcement learning-based face tracking system for Zenbo Jr. As robots become increasingly integrated into daily life, human-robot interaction (HRI) has emerged as a critical research area, especially in adhering to social norms. This project combines deep learning with visual servo systems to enhance the accuracy and response speed of robotic face tracking. The primary objectives of the study include: adjusting robot movements based on face position, designing a virtual environment using OpenAI Gym that matches Zenbo Jr.’s perspective for model training, and testing and validating the model on Zenbo Jr. The research methods involve designing the virtual environment, establishing reward functions, and constructing and training the DQN model. Experimental results compare the training outcomes of Q-Learning and Deep Q-Learning, confirming that Q-Learning leads to an excessively large $\mathbf{Q}$-table in scenarios with numerous state-action combinations, making effective training impossible, whereas Deep Q-Learning achieves stable convergence and good results.