A Semi-Parallel Multi-Timescale Reinforcement Learning Method for Coordinated Motion Planning and Hydraulic Control of Mobile Manipulators

For mobile manipulators, the global motion planning and hydraulic-system dynamic response operate on inherently mismatched timescales, making their coordinated control challenging. This paper proposes DASMT-PPO, a multi-timescale proximal policy optimization method with a dual-actor semi-parallel architecture. The proposed method preserves necessary information exchange across hierarchical levels while assigning distinct optimization roles to global task planning and hydraulic dynamic control. Specifically, the two actors optimize policies for global task planning and the rapid dynamic response of the hydraulic system, respectively. The multi-timescale mechanism updates the two policies at their corresponding timescales, thereby helping to reduce gradient interference between different control objectives. Meanwhile, the semi-parallel information flow maintains consistency between local dynamic corrections and global task objectives. Simulation results show that, compared with PPO, DASMT-PPO increases the task success rate from 82.5% to 94.0%, an improvement of 11.5 percentage points, and reduces the collision rate from 12.5% to 4.0%, a reduction of 8.5 percentage points. It also achieves higher episodic cumulative return and terminal control accuracy than SAC and TD3. The ablation results indicate that both the semi-parallel information flow and the combination of the dual-actor architecture with the multi-timescale mechanism contribute to improved algorithm performance.