Gunner: playing FPS game Doom with scalable deep reinforcement learning

Since the introduction of the Deep Q-Network (DQN) algorithm, the reinforcement learning research community has advanced rapidly. Much of the research has focused on the Atari 2600, which uses 2D environments to simulate aspects of the real world. However, 3D game environments are more advantageous in this regard, as they more closely approximate real-world settings. In this work, we focus on training an agent to play the deathmatch scenario in the 3D first-person shooter (FPS) game Doom. A key challenge lies in accurately identifying objects and retaining their spatial positions. We have developed a new agent, named Gunner, trained with a deep reinforcement learning algorithm that incorporates Residual and Long Short-Term Memory (LSTM) blocks for object recognition and memory. In addition, we further enhance the performance of Gunner by integrating Dueling Networks and Noisy Networks. During training, we adopted the AdamW optimizer as the default configuration. All training and testing experiments were conducted on the VizDoom platform. We evaluated the effectiveness of the algorithm components in the deathmatch scenario and further validated the generalization ability of the algorithm in two additional scenarios: “Health Gathering Supreme” and “Defend the Center”. Our experimental results demonstrate that Gunner outperforms several other reinforcement learning algorithms in the FPS game Doom.