Advantage-guided asynchronous advantage actor-critic for robust robot environment interaction in dynamic tasks

Dynamic and uncertain robot environments impose strict requirements on decision-making efficiency, convergence stability, and policy robustness. This study proposes an advantage-guided asynchronous advantage actor-critic framework for robot–environment interaction optimization. The method integrates multi-step advantage estimation, entropy-regularized exploration, smoothed value updating, and asynchronous multi-worker learning within a robot-oriented A3C structure. Experiments were conducted in Isaac Gym/Gazebo environments using a differential mobile robot and a Franka Emika Panda manipulator across path tracking, dynamic obstacle avoidance, and target approach/grasping tasks. At 15,000 training iterations, the improved A3C achieved a cumulative reward of − 1 and a strategy variance of 0.22, compared with − 2 and 0.35 for the standard A3C. With 32 workers, interaction throughput reached 4,810 steps/s, compared with 2,670 steps/s for PPO and 2,110 steps/s for SAC. At a noise level of 0.30, the performance retention rate remained 0.880, while the mission success rate remained 85.0% under the highest disturbance level. The average action latency was 27.84 ms. These results indicate that the proposed method improves interaction efficiency, convergence stability, and disturbance robustness in dynamic robotic tasks.