Multi‐Agent Reinforcement Learning for Cyber Defence Transferability and Scalability

ABSTRACT Reinforcement learning (RL) has shown to be effective for simple automated cyber defence (ACD) type tasks. However, there are limitations to these approaches that prevent them from being deployed onto real‐world hardware. Trained RL policies will often have limited transferability across even small changes to the environment setup. Instability during training can prevent optimal learning, a problem that only increases as the environment scales and grows in complexity. This work looks at addressing these limitations with a zero‐shot transfer approach based on multi‐agent RL. This is achieved by partitioning the task into smaller network machine subtasks, where agents learn the solution to the local problem. These local agents are independent of the network scale and can therefore be transferred to larger networks by mapping the agents to machines in the new network. Initial experiments show that this transfer method is effective for direct application to a number of ACD tasks. It is also shown that its performance is robust to changes in network activity, attack scenario and reduces the effects of network scale on performance.