Suboptimality of Constrained Action Adversarial Cyber-Physical Games

Abstract Analysing complex cyber-physical systems using established game-theoretic tools poses significant challenges due to the nonlinear dynamics inherent to such systems. To address this, we leverage multi-agent reinforcement learning (MARL) to study the impact constrained action spaces have on a player’s ability to uncover optimal strategies in a system governed by adversarial nonlinear dynamics. The system is posed as a dynamic, two-player, zero-sum game with elements of adversarial decision-making and resource competition, making it suitable for a variety of cyber-security, business, and military scenarios. Comparing player strategies over an ensemble of different action spaces suggests that MARL converges to an approximate $$\epsilon $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>ϵ</mml:mi> </mml:math> -Nash equilibrium even under constraints. In addition, numerical results reveal agreement between MARL solutions and our theoretical understanding of the problem, offering insight into action space selection for this adversarial game.