Deep Reinforcement Learning in Wargaming

We investigate the potential of deep reinforcement learning (RL) for the development of autonomous wargaming agents. We discuss the relevant characteristics of wargaming environments for the design of learning systems, the choice of learning framework, and algorithms. While deep RL has been demonstrated to achieve superhuman levels in various games, we argue that these findings can be only partially transferred to practical wargaming. This is due to real-world limitations such as the availability of financial and data resources, but also architectural system requirements that might rarely be satisfied in the wargaming area. The high degree of realism of modern warfare simulation environments is often accompanied by a system latency that entails impractical training times. For an empirical analysis, we adapt various deep RL techniques to the popular Command: Modern Operations simulation environment, providing a proof-of-concept for deep RL training applications in this environment.