Game-Theoretic Defense Policy for Network Security Against Intelligent Adversary

The rapid evolution of IT infrastructure and networked systems has increased their susceptibility to sophisticated and intelligent cyber threats. Despite advancements in attack detection, adversaries continuously refine their strategies, exploiting vulnerabilities with growing complexity. In this paper, we model the dynamic interaction between a defender and an intelligent adversary as a two-player zero-sum game. The defender’s partial observability of the adversary and network state is represented using a partially observable Markov decision process (POMDP). We develop a recursive method to compute the posterior distribution of network compromises based on incomplete observations of network states and no access to adversarial actions. An optimal minimum mean square error (MMSE) estimator leverages this posterior for the recursive estimation of network compromises. To ensure the defender follows the Nash equilibrium, where neither player has the incentive to deviate, our automated defense policy employs the Nash strategy based on the optimal MMSE estimate of the network state. Two evaluation metrics are introduced to assess the policy’s effectiveness: expected mean square error and expected policy misalignment. Simulation results show improved defense effectiveness over static or non-strategic automated policies, demonstrating the advantages of strategic decision-making in network security.