A Hybrid SAC Algorithm: Advantage-Guided Soft Actor-Critic

This paper presents a hybrid improvement algorithm that combines Soft Actor-Critic (SAC) and Advantage Actor-Critic (A2C) to enhance the performance of reinforcement learning in both continuous and discrete action spaces. The A2C algorithm improves the policy through the advantage function, but issues with convergence speed and stability persist in high-dimensional continuous action spaces. On the other hand, SAC enhances exploration by maximizing policy entropy, but it is highly dependent on the policy structure. The proposed algorithm integrates the maximum entropy regularization term from SAC with the advantage function from A2C, forming a hybrid objective function with advantage estimation. Experimental results in typical environments demonstrate that this method not only improves the convergence speed but also enhances the stability of the policy.