Optimized Deep Reinforcement Learning for Dual-Task Control in Deep-Sea Mining: Path Following and Obstacle Avoidance

This study investigates the dual-task control challenge of path following and obstacle avoidance for deep-sea mining robots operating in complex, unstructured environments. To address the limitations of traditional training strategies, we propose an optimized training framework that integrates environmental design enhancements and algorithmic advancements. Specifically, we develop a Dual-Task Training Environment by combining the Random Obstacle Environment with a newly proposed Obstructed Path Environment, ensuring a balanced learning approach. While agents trained solely in the Random Obstacle Environment exhibit unilateral obstacle avoidance strategies and achieve a 0% success rate in randomized obstacle scenarios, those trained in the Dual-Task Environment demonstrate 85.4% success under identical test conditions and acquire more complex bilateral avoidance strategies. Additionally, we introduce a Dynamic Multi-Step Update mechanism, which integrates immediate rewards with long-term returns to enhance deep reinforcement learning (Twin Delayed Deep Deterministic Policy Gradient, TD3) performance without increasing computational complexity. Under the optimal multi-step setting (n = 5), the Dynamic Multi-Step Update mechanism significantly improves path following accuracy, reducing trajectory deviations to 0.128 m on straight paths and 0.195 m on S-shaped paths, while achieving nearly 100% success in multi-directional obstacle avoidance tests. These improvements collectively enhance the adaptability, robustness, and operational performance of deep-sea mining robots, advancing intelligent control strategies for autonomous deep-sea exploration and resource extraction.