An Expert Experience-Enhanced Security Control Approach for AUVs of the Underwater Transportation Cyber-Physical Systems

By combining transportation information with physical elements, transportation cyber-physical systems (T-CPS) take advantage of the strengths of information technology and show great potential in terms of efficiency, safety, and control. T-CPS covers land, air, and underwater domains involving vehicles, drones, and autonomous underwater vehicles (AUVs), facilitating our lives and creating productivity. However, underwater T-CPS faces greater difficulties and challenges than the first two areas. On the one side, underwater equipment is generally expensive and thus requires a high level of safety. On the other side, the complexity of the marine environment causes uncertainty in the control. To address these challenges, this paper proposes an expert experience-enhanced control approach designed to enhance AUV reliability and safety. Firstly, we model AUV cluster control, including the complex underwater environment and cooperative control strategy, and refine this problem into a Markov decision problem (MDP) model based on the leader-follower strategy. Subsequently, a multi-agent reinforcement learning cluster control algorithm is developed on the framework of Centralized Training Distributed Execution (CTDE) to improve the learning and exploration capabilities of AUVs. Finally, we propose an expert experience-enhanced strategy that reduces the impact of non-smooth environments and also ameliorates the limitation of relying exclusively on rule-based experience. Experiments compare the linear and triangular AUV formation control tasks, and the proposed approach shows promising superiority and possesses sound stability in dynamically changing environments.