Dynamic Generation Technology of Network Scene Based on Proximal Policy Optimization

The cyber range can realistically simulate a variety of real network scenarios, and provide a safe and controllable platform for researchers for network security research and talent training. However, the static preset scene mode is widely used in the mainstream cyber range, which has three prominent limitations. Firstly, the manually designed fixed topology scene has low reuse rate, which is difficult to meet the differentiated training needs. Secondly, scenario adjustment relies on expert experience, which is slow and costly. Third, the generated scenarios are not diverse enough to support incremental capacity building. Therefore, based on the proximal policy optimization (PPO) algorithm, this paper models the existing network scenario, and designs the action space and reward function of PPO. The PPO algorithm dynamically generates multiple new scenes that are equivalent to the original scene evaluation by inputting the existing static scene template. Finally, the effectiveness and superiority of the proposed algorithm are verified by comparing with the traditional genetic algorithm.