A Hierarchical Reinforcement Learning Method for Intelligent Decision-Making in Joint Operations of Sea–Air Unmanned Systems

To address the challenges of intelligent decision-making in complex and high-dimensional state–action spaces during joint operations simulations of sea–air unmanned systems, an end-to-end intelligent decision-making scheme is proposed. Initially, a highly versatile hierarchical intelligent decision-making method is designed for sea–air joint operations simulation scenarios. Subsequently, an approach combining intrinsic and extrinsic rewards is adopted to structurally mitigate the adverse effects of sparse rewards. Following this, a prominence detection method and a repetition penalty filtering method are devised, leading to the development of a hierarchical reinforcement learning algorithm based on a two-tier screening approach for potential subgoals. Finally, the feasibility of the proposed method is validated through ablation experiments and visualized simulation studies. Simulation results demonstrate that the presented method offers some reference value for research in intelligent decision-making for unmanned operations and can be applied to innovative studies in related response strategies.