Reward Design in Self-Play Reinforcement Learning for Dou Shou Qi: Effects on Learning Efficiency and Behavioral Characteristics

Reward Design helps to motivate the learning process and modify the behavior of agents in reinforcement learning through intermediate feedback. However, the impact of reward-design parameters on the learning efficiency and behavior of competitive self-play board games has not been fully explored. Research on Reward Shaping and Reward-Parameter Strength in Self-Play Reinforcement Learning for Dou Shou Qi. Controlled reward-ablation experiments were conducted using Proximal Policy Optimization with a shared 64-channel, four-block residual network on a 7×9 board. Three independent seeds were used to train the agents for 1,000,000 environment steps in total. Compare the experimental results of reward shaping and no shaping, vary the shaping coefficient from 0.25 to 1.0, and change the repetition-penalty coefficient between 0.002 and 0.05. Learning efficiency is the number of environment steps needed to reach a stable performance of 0.95, and behavior indicators include episode length, repetition rate, policy entropy, and the final evaluation win rate. Under the reward-shaping condition, the sustained high-performance group needed 34.7% fewer environment steps in IQM than the non-shaping group. Increased shaping strength did not lead to further improvements in learning efficiency, and behavior was altered by changing reward parameters. Therefore, the design of rewards should be judged based on learning efficiency and behavioral changes.