Soccer Match Strategy Scheduling Based on NAO Robots

: Intelligent robots especially NAO robots are expected to play an important role in various vertical industries and robot soccer match is an effective method for the research of multi-agent systems (MAS). Due to the highly dynamic and complex environment of the football field and the requirements for the real-time action of robot players, the optimization of robot strategy scheduling is recognized as an essential component of robot soccer match. Nevertheless, state-of-the-art decision-making on robot action scheduling can suffer from the processing of huge amount of date and the interaction between robots. To handle this issue, this paper proposes a robot soccer match strategy scheduling scheme, based on reinforcement learning (RL). We propose to formulate the decision-making problem of robot action scheduling as a Markov decision process (MDP), i.e. the basis of developing algorithms of deep Q-network (DQN) and proximal policy optimization (PPO). In order to strengthen the cooperation between robot players, we apply client-server (C-S) architecture to achieve efficient communication and joint scheduling between robot players. The simulation results confirm that the proposed algorithm is cable of learning a scheduling policy in the presence of a robot soccer match and significantly optimizing the robot action decision-making problem.