A Human Multi-Agent Teaming Testbed, Escape Room Simulation

Scalability of multi-agent systems is quickly becoming a crucial area of research as artificial agents become more capable and sophisticated. This is particularly relevant in the context of increasingly ubiquitous AI and automation. A key challenge to furthering research in the area of human-multi-agent teams is having suitable test domains that are simple enough to allow thorough analysis, yet rich enough to foster an appropriate level of human-agent interaction. We present a new open-source test domain designed and developed to explore the dynamics of human interactions with multi-agent systems and the factors that enable and inhibit scalability.Our desire to understand and model the factors that influence scalability and related measures led us to search for a testbed simulation that could provide a suite of capabilities. First, it needed to model a specific type of activity, where the human owns the mission goals and coordinates the efforts of multiple agents to achieve a common goal. Second, we needed to be able to control the interaction and observability of the agents. Lastly, we needed to be able to adjust the autonomy of the agents. After researching available testbeds, we did not find many simulations that met these criteria, were readily available, open-source, easy to use, and most importantly, provided the desired human interaction dynamics. As such, we developed a suitable simulation testbed to explore measures of scalability and how both the characteristics of the technology and the human users affect the scalability of human multi-agent teams.The mission scenario we found that exemplifies the collaborative teamwork dynamics we are interested in is the escape room; a cooperative human team game that became popular in the last couple of decades. The escape room models the aspects of teamwork we were looking for: exploration, partial knowledge, distributed execution, interdependent decision-making, and puzzle-solving. Our escape room simulation is a single-player game where the human player directs a team of agents to explore several rooms to collect keys to open doors. The simulation can be simple with just a few keys and doors or increasingly complex with several layers of keys and doors to solve.In the escape room simulation, we can control the two main drivers of scalability: interaction and automation. The testbed is instrumented to measure both, as well as a host of additional relevant measures for analyzing scalability. We explain how the testbed supports several of the more popular models of scalability as well as our own variant. The simulation was developed using the p5 JavaScript library and the p5.js web editor. The source code is freely accessible online for tailoring the environment and modifying agent capabilities. With the increase of AI and agent capabilities and the desire to employ ever larger multi-agent teams, it is important to have a simulation to test out theories to better understand what influences, constrains and enables effective human multi-agent teaming. Our simulation provides a valuable tool for researchers and practitioners in this evolving field.