Integrating model predictive control and reinforcement learning for robust Six-DOF AUV trajectory tracking

To address model uncertainties, ocean-current disturbances, and real-time control requirements in six-degree-of-freedom AUV trajectory tracking, this paper proposes a residual-learning-enhanced MPC–PID hierarchical control framework. An outer-loop kinematic model predictive controller generates constraint-aware velocity references, while an inner-loop inverse-dynamics PID controller provides nominal dynamic compensation and fast velocity tracking. A reinforcement-learning-based compensator generates bounded residual actions to mitigate tracking errors caused by unmodeled dynamics and external disturbances without replacing the model-based controller. The proposed method is evaluated on a six-degree-of-freedom AUV simulation platform using three representative trajectories under nominal and random ocean-current conditions. Compared with representative model-based and learning-based controllers, RLMPC achieves a better overall trade-off among position tracking, attitude stability, disturbance rejection, control effort, and actuator constraint satisfaction. An additional comparison with MPC–PID–DOB shows that conventional observer-based compensation improves the robustness of MPC–PID, whereas residual learning provides more balanced performance across the tested scenarios. Under random disturbances, RLMPC reduces position and attitude tracking errors by up to 36.2% and 47.1%, respectively, relative to MPC–PID. The controller also maintains millisecond-level computation time, demonstrating its real-time computational feasibility.